Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 941197 Details for
Bug 1146648
master domain down after upgrade, tracebacks in vdsm.log
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
engine log
engine.log (text/plain), 4.51 MB, created by
Petr Beňas
on 2014-09-25 16:42:48 UTC
(
hide
)
Description:
engine log
Filename:
MIME Type:
Creator:
Petr Beňas
Created:
2014-09-25 16:42:48 UTC
Size:
4.51 MB
patch
obsolete
>2014-09-09 15:24:44,678 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-09 15:24:44,678 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-09 15:24:44,679 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-09 15:24:44,691 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-09 15:24:44,691 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-09 15:24:44,691 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-09 15:24:44,692 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-09 15:24:44,692 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-09 15:24:44,692 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-09 15:24:44,693 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-09 15:24:44,693 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-09 15:24:44,695 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-09 15:24:44,696 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-09 15:24:44,697 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-09 15:24:44,697 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-09 15:24:44,697 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-09 15:24:44,704 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-09 15:24:44,705 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-09 15:24:44,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-09 15:24:44,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-09 15:24:44,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-09 15:24:44,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-09 15:24:44,706 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-09 15:24:44,707 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-09 15:24:44,708 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-09 15:24:44,708 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-09 15:24:44,708 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-09 15:24:44,708 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-09 15:24:44,714 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-09 15:24:44,714 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-09 15:24:44,714 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-09 15:24:44,714 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-09 15:24:44,714 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-09 15:24:44,715 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-09 15:24:44,716 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-09 15:24:44,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-09 15:24:44,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-09 15:24:44,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-09 15:24:44,717 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-09 15:24:44,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-09 15:24:44,725 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-09 15:24:44,726 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-09 15:24:44,727 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-09 15:24:44,728 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-09 15:25:11,957 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) Start initializing Backend >2014-09-09 15:25:12,607 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 51) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-09 15:25:12,795 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) Running ovirt-engine 3.5.0-0.10.master.el6ev >2014-09-09 15:25:12,795 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 51) Start initializing dictionaries >2014-09-09 15:25:12,804 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 51) Finished initializing dictionaries >2014-09-09 15:25:12,804 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Start initializing AuditLogCleanupManager >2014-09-09 15:25:12,805 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:25:12,832 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Finished initializing AuditLogCleanupManager >2014-09-09 15:25:12,833 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Start initializing CommandEntityCleanupManager >2014-09-09 15:25:12,833 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:25:12,840 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 51) Finished initializing CommandEntityCleanupManager >2014-09-09 15:25:12,844 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 51) Start initializing TagsDirector >2014-09-09 15:25:12,859 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 51) Tag root added to tree >2014-09-09 15:25:12,877 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 51) Finished initializing TagsDirector >2014-09-09 15:25:12,877 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 51) Start initializing IsoDomainListSyncronizer >2014-09-09 15:25:12,884 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 51) Finished initializing IsoDomainListSyncronizer >2014-09-09 15:25:12,926 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) Completed initializing handlers >2014-09-09 15:25:12,953 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 51) Start initializing ErrorTranslatorImpl >2014-09-09 15:25:12,967 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 51) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-09 15:25:12,968 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 51) Finished initializing ErrorTranslatorImpl >2014-09-09 15:25:12,968 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 51) Start initializing ErrorTranslatorImpl >2014-09-09 15:25:12,969 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 51) Finished initializing ErrorTranslatorImpl >2014-09-09 15:25:12,969 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 51) Mark incomplete jobs as UNKNOWN >2014-09-09 15:25:13,020 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 51) Start initializing JobRepositoryCleanupManager >2014-09-09 15:25:13,020 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 51) Finished initializing JobRepositoryCleanupManager >2014-09-09 15:25:13,021 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 51) Start initializing AutoRecoveryManager >2014-09-09 15:25:13,022 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 51) Finished initializing AutoRecoveryManager >2014-09-09 15:25:13,023 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 51) Start initializing ExecutionMessageDirector >2014-09-09 15:25:13,025 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 51) Finished initializing ExecutionMessageDirector >2014-09-09 15:25:13,136 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Loading extension 'builtin-authn-internal' >2014-09-09 15:25:13,138 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Extension 'builtin-authn-internal' loaded >2014-09-09 15:25:13,139 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Loading extension 'internal' >2014-09-09 15:25:13,147 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Extension 'internal' loaded >2014-09-09 15:25:13,149 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Initializing extension 'builtin-authn-internal' >2014-09-09 15:25:13,149 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Extension 'builtin-authn-internal' initialized >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Initializing extension 'internal' >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Extension 'internal' initialized >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Start of enabled extensions list >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:25:13,150 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 51) End of enabled extensions list >2014-09-09 15:25:13,275 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 48) Start initializing DbUserCacheManager >2014-09-09 15:25:13,278 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 48) Finished initializing DbUserCacheManager >2014-09-09 15:25:13,332 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 48) Initialization of AsyncTaskManager completed successfully. >2014-09-09 15:25:13,334 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Start initializing ResourceManager >2014-09-09 15:25:13,394 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-09 15:25:13,579 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Finished initializing ResourceManager >2014-09-09 15:25:13,584 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 48) Initialization of OvfDataUpdater completed successfully. >2014-09-09 15:25:13,585 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start scheduling to enable vds load balancer >2014-09-09 15:25:13,585 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished scheduling to enable vds load balancer >2014-09-09 15:25:13,585 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start HA Reservation check >2014-09-09 15:25:13,586 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished HA Reservation check >2014-09-09 15:25:13,600 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-09 15:25:13,610 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init VM custom properties utilities >2014-09-09 15:25:13,611 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init device custom properties utilities >2014-09-09 15:25:13,621 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initializing Scheduling manager >2014-09-09 15:25:13,649 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 256 >2014-09-09 15:25:13,684 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) External scheduler disabled, discovery skipped >2014-09-09 15:25:13,684 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initialized Scheduling manager >2014-09-09 15:25:13,684 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) Initializing DWH Heart Beat >2014-09-09 15:25:13,692 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) DWH Heart Beat initialized >2014-09-09 15:28:26,703 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:28:26,707 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:28:38,980 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-09 15:28:38,996 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-09 15:28:39,023 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:28:39,023 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:28:58,966 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginAdminUserCommand internal: false. >2014-09-09 15:28:58,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: UNASSIGNED not exist in string table >2014-09-09 15:28:58,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-09-09 15:28:58,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-09-09 15:28:58,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-09-09 15:28:58,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-09 15:28:58,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-09 15:28:58,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-09 15:28:58,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-09 15:28:58,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-09 15:28:58,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: DWH_STOPPED not exist in string table >2014-09-09 15:28:58,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: DWH_STARTED not exist in string table >2014-09-09 15:28:58,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: DWH_ERROR not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-09 15:28:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-09 15:28:58,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-09-09 15:28:58,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:28:58,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-09-09 15:28:58,992 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:28:59,494 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-09 15:29:04,235 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2014-09-09 15:31:48,934 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-09 15:31:48,934 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-09 15:31:48,935 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-09 15:31:48,935 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-09 15:31:48,935 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-09 15:31:48,944 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-09 15:31:48,948 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-09 15:31:48,948 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-09 15:31:48,948 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-09 15:31:48,949 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-09 15:31:48,950 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-09 15:31:48,951 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-09 15:31:48,960 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-09 15:31:48,961 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-09 15:31:48,962 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-09 15:31:48,963 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-09 15:31:48,971 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-09 15:31:48,972 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-09 15:31:48,972 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-09 15:31:48,972 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-09 15:31:59,286 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Start initializing Backend >2014-09-09 15:31:59,698 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ajp-/127.0.0.1:8702-1) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-09 15:31:59,818 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Running ovirt-engine 3.5.0-0.10.master.el6ev >2014-09-09 15:31:59,818 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Start initializing dictionaries >2014-09-09 15:31:59,826 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Finished initializing dictionaries >2014-09-09 15:31:59,826 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing AuditLogCleanupManager >2014-09-09 15:31:59,826 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:31:59,844 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing AuditLogCleanupManager >2014-09-09 15:31:59,845 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing CommandEntityCleanupManager >2014-09-09 15:31:59,845 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:31:59,850 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing CommandEntityCleanupManager >2014-09-09 15:31:59,853 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Start initializing TagsDirector >2014-09-09 15:31:59,866 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Tag root added to tree >2014-09-09 15:31:59,884 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Finished initializing TagsDirector >2014-09-09 15:31:59,885 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Start initializing IsoDomainListSyncronizer >2014-09-09 15:31:59,891 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Finished initializing IsoDomainListSyncronizer >2014-09-09 15:31:59,937 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Completed initializing handlers >2014-09-09 15:31:59,965 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl >2014-09-09 15:31:59,984 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-09 15:31:59,985 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl >2014-09-09 15:31:59,985 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl >2014-09-09 15:31:59,986 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl >2014-09-09 15:31:59,986 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Mark incomplete jobs as UNKNOWN >2014-09-09 15:32:00,029 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing JobRepositoryCleanupManager >2014-09-09 15:32:00,030 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing JobRepositoryCleanupManager >2014-09-09 15:32:00,030 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Start initializing AutoRecoveryManager >2014-09-09 15:32:00,034 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Finished initializing AutoRecoveryManager >2014-09-09 15:32:00,036 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Start initializing ExecutionMessageDirector >2014-09-09 15:32:00,038 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Finished initializing ExecutionMessageDirector >2014-09-09 15:32:00,155 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'builtin-authn-internal' >2014-09-09 15:32:00,158 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' loaded >2014-09-09 15:32:00,161 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'internal' >2014-09-09 15:32:00,164 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' loaded >2014-09-09 15:32:00,164 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'builtin-authn-internal' >2014-09-09 15:32:00,164 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' initialized >2014-09-09 15:32:00,165 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'internal' >2014-09-09 15:32:00,165 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' initialized >2014-09-09 15:32:00,166 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Start of enabled extensions list >2014-09-09 15:32:00,166 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:32:00,166 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:32:00,166 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) End of enabled extensions list >2014-09-09 15:32:00,508 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 44) Start initializing DbUserCacheManager >2014-09-09 15:32:00,512 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 44) Finished initializing DbUserCacheManager >2014-09-09 15:32:00,566 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 44) Initialization of AsyncTaskManager completed successfully. >2014-09-09 15:32:00,567 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 44) Start initializing ResourceManager >2014-09-09 15:32:00,676 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-09 15:32:00,815 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 44) Finished initializing ResourceManager >2014-09-09 15:32:00,820 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 44) Initialization of OvfDataUpdater completed successfully. >2014-09-09 15:32:00,821 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Start scheduling to enable vds load balancer >2014-09-09 15:32:00,821 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Finished scheduling to enable vds load balancer >2014-09-09 15:32:00,821 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Start HA Reservation check >2014-09-09 15:32:00,821 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Finished HA Reservation check >2014-09-09 15:32:00,839 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-09 15:32:00,855 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 44) Init VM custom properties utilities >2014-09-09 15:32:00,856 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 44) Init device custom properties utilities >2014-09-09 15:32:00,866 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Initializing Scheduling manager >2014-09-09 15:32:00,887 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 65 >2014-09-09 15:32:00,951 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) External scheduler disabled, discovery skipped >2014-09-09 15:32:00,951 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 44) Initialized Scheduling manager >2014-09-09 15:32:00,951 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 44) Initializing DWH Heart Beat >2014-09-09 15:32:00,960 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 44) DWH Heart Beat initialized >2014-09-09 15:32:02,460 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [3225340d] Running command: LogoutUserCommand internal: false. >2014-09-09 15:32:02,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: UNASSIGNED not exist in string table >2014-09-09 15:32:02,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-09-09 15:32:02,554 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-09-09 15:32:02,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-09-09 15:32:02,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-09 15:32:02,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-09 15:32:02,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-09 15:32:02,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-09 15:32:02,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-09 15:32:02,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: DWH_STOPPED not exist in string table >2014-09-09 15:32:02,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: DWH_STARTED not exist in string table >2014-09-09 15:32:02,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: DWH_ERROR not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,559 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-09 15:32:02,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-09-09 15:32:02,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:32:02,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-09-09 15:32:02,561 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:32:02,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [3225340d] Correlation ID: 3225340d, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-09 15:32:03,072 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-9) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-09 15:32:03,094 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-9) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-09 15:32:03,428 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-09 15:32:03,428 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-09 15:32:16,342 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-13) Running command: LoginAdminUserCommand internal: false. >2014-09-09 15:32:16,770 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-13) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-09 15:32:20,519 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginUserCommand internal: false. >2014-09-09 15:36:57,175 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-1) [14519643] CanDoAction of action AddVds failed. Reasons:VAR__ACTION__ADD,VAR__TYPE__HOST,$server dell-r210ii-16.rhev.lab.eng.brq.redhat.com,VDS_CANNOT_INSTALL_EMPTY_PASSWORD >2014-09-09 15:37:08,173 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-4) [3307426a] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-09 15:37:08,500 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:37:08,530 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-09 15:37:08,540 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] START, RemoveVdsVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488), log id: 712d6003 >2014-09-09 15:37:08,541 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] FINISH, RemoveVdsVDSCommand, log id: 712d6003 >2014-09-09 15:37:08,545 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] START, AddVdsVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488), log id: 4c5eff5a >2014-09-09 15:37:08,545 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] AddVds - entered , starting logic to add VDS 14369bd0-907d-40d7-be1e-bd35af6d1488 >2014-09-09 15:37:08,548 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] AddVds - VDS 14369bd0-907d-40d7-be1e-bd35af6d1488 was added, will try to add it to the resource manager >2014-09-09 15:37:08,553 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [5cb487dd] Entered VdsManager constructor >2014-09-09 15:37:08,580 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [5cb487dd] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-09 15:37:08,748 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [5cb487dd] VDS 14369bd0-907d-40d7-be1e-bd35af6d1488 was added to the Resource Manager >2014-09-09 15:37:08,757 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [5cb487dd] FINISH, AddVdsVDSCommand, log id: 4c5eff5a >2014-09-09 15:37:08,774 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [5cb487dd] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-09 15:37:08,848 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [5cb487dd] Correlation ID: 3307426a, Job ID: a335eda2-f1e7-4c42-a7c8-1714f9c188bd, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-09 15:37:08,895 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:37:08,931 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:37:08,938 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] Before Installation host 14369bd0-907d-40d7-be1e-bd35af6d1488, rack >2014-09-09 15:37:08,956 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 47873fe0 >2014-09-09 15:37:09,002 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] FINISH, SetVdsStatusVDSCommand, log id: 47873fe0 >2014-09-09 15:37:09,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-4) [461323e4] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 3f:fe:39:f2:21:7b:9c:a0:53:49:a6:35:b0:6c:4a:ff >2014-09-09 15:37:09,102 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-4) [461323e4] Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 3f:fe:39:f2:21:7b:9c:a0:53:49:a6:35:b0:6c:4a:ff. >2014-09-09 15:37:09,123 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-4) [461323e4] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-09 15:37:09,123 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (org.ovirt.thread.pool-6-thread-4) Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' is missing, creating >2014-09-09 15:37:09,367 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-4) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-09 15:37:10,269 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-09 15:37:10,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-09 15:37:10,312 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-09 15:37:10,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-09 15:37:10,340 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-09 15:37:10,362 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-09 15:37:10,363 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-09 15:37:10,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-09 15:37:10,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-09 15:37:10,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-09 15:37:10,389 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump unsupported >2014-09-09 15:37:10,400 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump unsupported. >2014-09-09 15:37:10,417 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140909153709-84s9io.log' >2014-09-09 15:37:10,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140909153709-84s9io.log'. >2014-09-09 15:37:10,535 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-09 15:37:10,538 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 15:37:10,658 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-09 15:37:10,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-09 15:37:10,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-09 15:37:10,722 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-09 15:37:10,747 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-09 15:37:10,756 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-09 15:37:10,757 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-09 15:37:10,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-09 15:37:10,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 15:37:10,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 15:37:10,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-09 15:37:10,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-09 15:37:11,756 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Initializing Host: rack >2014-09-09 15:37:18,291 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 15:37:18,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 15:37:18,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-09 15:37:18,387 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-09 15:37:21,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-09 15:37:21,179 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-09 15:37:22,541 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-09 15:37:22,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-09 15:37:22,556 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-09 15:37:22,581 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-09 15:37:23,418 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-09 15:37:23,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-09 15:37:23,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-09 15:37:23,718 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-09 15:37:25,687 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-09 15:37:25,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-09 15:37:25,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909153725-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-461323e4.log' >2014-09-09 15:37:25,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909153725-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-461323e4.log'. >2014-09-09 15:37:25,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-09 15:37:25,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 461323e4, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-09 15:37:26,129 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 15:37:26,476 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-4) [461323e4] Engine managed to communicate with VDSM agent on host rack >2014-09-09 15:37:27,099 WARN [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-4) [461323e4] The lastClientInterface ovirtmgmt of host rack is not a valid interface for the mangement network. If the interface is a bridge, it should be torn-down manually. >2014-09-09 15:37:27,105 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] Host installation failed for host 14369bd0-907d-40d7-be1e-bd35af6d1488, rack.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:206) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view6.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: lastClientIface ovirtmgmt is not a valid interface for management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.findNicToSetupManagementNetwork(NetworkConfigurator.java:170) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:74) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:198) [bll.jar:] > ... 70 more > >2014-09-09 15:37:27,124 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7a39ef2a >2014-09-09 15:37:27,148 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] FINISH, SetVdsStatusVDSCommand, log id: 7a39ef2a >2014-09-09 15:37:27,204 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-4) [461323e4] Correlation ID: 461323e4, Job ID: a335eda2-f1e7-4c42-a7c8-1714f9c188bd, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. Failed to configure management network on the host. >2014-09-09 15:37:27,214 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-4) [461323e4] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:40:00,108 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-71) Autorecovering 1 hosts >2014-09-09 15:40:00,109 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-71) Autorecovering hosts id: 14369bd0-907d-40d7-be1e-bd35af6d1488, name : rack >2014-09-09 15:40:00,121 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:40:00,133 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 15:40:00,133 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 15:40:00,133 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 15:40:00,152 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 120bd141 >2014-09-09 15:40:00,193 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] FINISH, SetVdsStatusVDSCommand, log id: 120bd141 >2014-09-09 15:40:00,329 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 15:40:00,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-71) [240b41f8] Correlation ID: 240b41f8, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-09 15:40:00,347 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-71) [240b41f8] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:40:03,178 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488]), log id: 5100fff0 >2014-09-09 15:40:03,248 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GetHardwareInfoVDSCommand, log id: 5100fff0 >2014-09-09 15:40:03,384 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [6349b998] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:03,387 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [6349b998] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 69771b18 >2014-09-09 15:40:03,419 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [6349b998] FINISH, SetVdsStatusVDSCommand, log id: 69771b18 >2014-09-09 15:40:03,461 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [6349b998] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 15:40:03,505 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [6349b998] Correlation ID: 6349b998, Job ID: 25cb8a86-e0de-4ea3-8ad7-776f5ba47e04, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 15:40:03,538 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-87) [741656c7] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:03,790 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (DefaultQuartzScheduler_Worker-87) [15828890] Running command: UpdateVdsGroupCommand internal: true. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN >2014-09-09 15:40:03,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [15828890] Correlation ID: 15828890, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by system >2014-09-09 15:40:03,953 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) [15828890] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488]), log id: 428f21ef >2014-09-09 15:40:03,958 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) [15828890] FINISH, GetHardwareInfoVDSCommand, log id: 428f21ef >2014-09-09 15:40:04,065 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [75e70b98] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:04,067 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [75e70b98] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: c95f15f >2014-09-09 15:40:04,081 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [75e70b98] FINISH, SetVdsStatusVDSCommand, log id: c95f15f >2014-09-09 15:40:04,128 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [75e70b98] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 15:40:04,144 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-87) [66060f81] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:04,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [66060f81] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 15:40:04,217 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-87) [66060f81] START, FullListVdsCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488], vmIds=[e8715843-38c5-4188-b00e-b9ceadce2714]), log id: f7bb3dd >2014-09-09 15:40:04,232 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-87) [66060f81] FINISH, FullListVdsCommand, return: [{guestFQDN=, acpiEnable=true, emulatedMachine=rhel6.5.0, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, vmId=e8715843-38c5-4188-b00e-b9ceadce2714, memGuaranteedSize=6100, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=Westmere, smp=1, pauseCode=NOERR, bootMenuEnable=false, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=6100, smpCoresPerSocket=1, vmName=341, nice=0, username=, exitMessage=User shut down from within the guest, pid=29513, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=-1, smartcardEnable=false, clientIp=, exitCode=0, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=ovirtmgmt, exitReason=7, status=Down, timeOffset=7200, maxVCpus=16, guestIPs=, devices=[Ljava.lang.Object;@73f10c54, display=qxl}], log id: f7bb3dd >2014-09-09 15:40:04,234 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) [66060f81] Importing VM 341 as external-341, as it is running on the on Host, but does not exist in the engine. >2014-09-09 15:40:04,639 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-87) [73ac8439] Lock Acquired to object EngineLock [exclusiveLocks= key: external-341 value: VM_NAME >, sharedLocks= ] >2014-09-09 15:40:04,774 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-87) [73ac8439] Running command: AddVmFromScratchCommand internal: true. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-09-09 15:40:05,899 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [73ac8439] Correlation ID: 73ac8439, Call Stack: null, Custom Event ID: -1, Message: VM external-341 was created by <UNKNOWN>. >2014-09-09 15:40:05,900 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-87) [73ac8439] Lock freed to object EngineLock [exclusiveLocks= key: external-341 value: VM_NAME >, sharedLocks= ] >2014-09-09 15:40:05,922 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-87) [151c998] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:05,929 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-87) [7adc4c5d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:40:05,932 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) [7adc4c5d] Host 14369bd0-907d-40d7-be1e-bd35af6d1488 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 15:41:16,318 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] Running command: SetupNetworksCommand internal: false. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-09 15:41:16,326 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] START, SetupNetworksVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em2 {id=c694570f-fe20-4bc0-a065-b27ab1f9e659, vdsId=14369bd0-907d-40d7-be1e-bd35af6d1488, name=em2, macAddress=d4:ae:52:c7:0d:8c, networkName=null, bondName=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null}, > em1 {id=235e3339-c91b-47fc-a126-36efad304f0a, vdsId=14369bd0-907d-40d7-be1e-bd35af6d1488, name=em1, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, bondName=null, bootProtocol=DHCP, address=null, subnet=null, gateway=null, mtu=0, bridged=true, speed=1000, type=2, networkImplementationDetails=null}], > removedNetworks=[ovirtmgmt], > removedBonds=[]), log id: 19556e72 >2014-09-09 15:41:16,333 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] FINISH, SetupNetworksVDSCommand, log id: 19556e72 >2014-09-09 15:44:16,479 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 15:44:16,486 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] Command SetupNetworksVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em2 {id=c694570f-fe20-4bc0-a065-b27ab1f9e659, vdsId=14369bd0-907d-40d7-be1e-bd35af6d1488, name=em2, macAddress=d4:ae:52:c7:0d:8c, networkName=null, bondName=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null}, > em1 {id=235e3339-c91b-47fc-a126-36efad304f0a, vdsId=14369bd0-907d-40d7-be1e-bd35af6d1488, name=em1, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, bondName=null, bootProtocol=DHCP, address=null, subnet=null, gateway=null, mtu=0, bridged=true, speed=1000, type=2, networkImplementationDetails=null}], > removedNetworks=[ovirtmgmt], > removedBonds=[]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 15:44:16,487 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (ajp-/127.0.0.1:8702-2) [6d52f989] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' (Failed with error VDS_NETWORK_ERROR and code 5022) >2014-09-09 15:45:00,021 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-51) Autorecovering 1 hosts >2014-09-09 15:45:00,021 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-51) Autorecovering hosts id: 14369bd0-907d-40d7-be1e-bd35af6d1488, name : rack >2014-09-09 15:45:00,027 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:45:00,028 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 15:45:00,028 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 15:45:00,028 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 15:45:00,038 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 32c6f377 >2014-09-09 15:45:00,061 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] FINISH, SetVdsStatusVDSCommand, log id: 32c6f377 >2014-09-09 15:45:00,113 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 15:45:00,119 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [4d71ac53] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:45:00,622 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 15:45:00,624 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:295) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:203) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:144) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 15:45:27,183 WARN [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (ajp-/127.0.0.1:8702-2) [66268694] CanDoAction of action SetupNetworks failed. Reasons:VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,VAR__HOST_STATUS__UP_MAINTENANCE_OR_NON_OPERATIONAL,ACTION_TYPE_FAILED_VDS_STATUS_ILLEGAL >2014-09-09 15:46:16,176 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-13) [7cdc5df5] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 15:46:16,177 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-13) [7cdc5df5] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_VDS_NOT_MATCH_VALID_STATUS >2014-09-09 15:46:16,177 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-13) [7cdc5df5] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 15:47:41,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-09 15:47:41,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-09 15:47:41,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-09 15:47:41,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-09 15:47:41,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-09 15:47:41,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-09 15:47:41,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-09 15:47:41,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-09 15:47:41,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-09 15:47:41,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-09 15:47:41,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-09 15:47:41,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-09 15:47:41,808 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-09 15:47:41,818 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_USER" is "engine". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-09 15:47:41,819 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-09 15:47:41,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-09 15:47:41,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-09 15:47:41,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-09 15:47:41,836 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-09 15:47:41,837 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-09 15:47:41,838 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-09 15:47:41,839 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USER" is "ovirt". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-09 15:47:41,840 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-09 15:47:41,841 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-09 15:47:41,841 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-09 15:47:50,521 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Start initializing Backend >2014-09-09 15:47:51,049 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 42) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-09 15:47:51,230 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Running ovirt-engine 3.5.0-0.10.master.el6ev >2014-09-09 15:47:51,231 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Start initializing dictionaries >2014-09-09 15:47:51,234 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 42) Finished initializing dictionaries >2014-09-09 15:47:51,235 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Start initializing AuditLogCleanupManager >2014-09-09 15:47:51,235 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:47:51,277 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Finished initializing AuditLogCleanupManager >2014-09-09 15:47:51,277 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Start initializing CommandEntityCleanupManager >2014-09-09 15:47:51,278 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-09 15:47:51,280 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 42) Finished initializing CommandEntityCleanupManager >2014-09-09 15:47:51,283 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Start initializing TagsDirector >2014-09-09 15:47:51,309 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Tag root added to tree >2014-09-09 15:47:51,337 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 42) Finished initializing TagsDirector >2014-09-09 15:47:51,337 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Start initializing IsoDomainListSyncronizer >2014-09-09 15:47:51,353 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 42) Finished initializing IsoDomainListSyncronizer >2014-09-09 15:47:51,432 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Completed initializing handlers >2014-09-09 15:47:51,475 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2014-09-09 15:47:51,498 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-09 15:47:51,500 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2014-09-09 15:47:51,500 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Start initializing ErrorTranslatorImpl >2014-09-09 15:47:51,500 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 42) Finished initializing ErrorTranslatorImpl >2014-09-09 15:47:51,500 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 42) Mark incomplete jobs as UNKNOWN >2014-09-09 15:47:51,561 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Start initializing JobRepositoryCleanupManager >2014-09-09 15:47:51,563 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 42) Finished initializing JobRepositoryCleanupManager >2014-09-09 15:47:51,563 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Start initializing AutoRecoveryManager >2014-09-09 15:47:51,565 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 42) Finished initializing AutoRecoveryManager >2014-09-09 15:47:51,567 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Start initializing ExecutionMessageDirector >2014-09-09 15:47:51,576 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 42) Finished initializing ExecutionMessageDirector >2014-09-09 15:47:51,754 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'builtin-authn-internal' >2014-09-09 15:47:51,764 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-internal' loaded >2014-09-09 15:47:51,766 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Loading extension 'internal' >2014-09-09 15:47:51,768 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'internal' loaded >2014-09-09 15:47:51,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'builtin-authn-internal' >2014-09-09 15:47:51,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'builtin-authn-internal' initialized >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Initializing extension 'internal' >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Extension 'internal' initialized >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Start of enabled extensions list >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-09 15:47:51,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 42) End of enabled extensions list >2014-09-09 15:47:51,982 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 47) Start initializing DbUserCacheManager >2014-09-09 15:47:51,991 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 47) Finished initializing DbUserCacheManager >2014-09-09 15:47:52,037 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 47) Initialization of AsyncTaskManager completed successfully. >2014-09-09 15:47:52,039 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 47) Start initializing ResourceManager >2014-09-09 15:47:52,204 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-09 15:47:52,313 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 47) Entered VdsManager constructor >2014-09-09 15:47:52,383 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 47) Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-09 15:47:52,544 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 47) VDS 14369bd0-907d-40d7-be1e-bd35af6d1488 was added to the Resource Manager >2014-09-09 15:47:52,580 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 47) Finished initializing ResourceManager >2014-09-09 15:47:52,586 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 47) Initialization of OvfDataUpdater completed successfully. >2014-09-09 15:47:52,587 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Start scheduling to enable vds load balancer >2014-09-09 15:47:52,588 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Finished scheduling to enable vds load balancer >2014-09-09 15:47:52,588 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Start HA Reservation check >2014-09-09 15:47:52,589 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Finished HA Reservation check >2014-09-09 15:47:52,602 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-09 15:47:52,610 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 47) Init VM custom properties utilities >2014-09-09 15:47:52,612 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 47) Init device custom properties utilities >2014-09-09 15:47:52,617 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Initializing Scheduling manager >2014-09-09 15:47:52,656 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 65 >2014-09-09 15:47:52,689 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) External scheduler disabled, discovery skipped >2014-09-09 15:47:52,689 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 47) Initialized Scheduling manager >2014-09-09 15:47:52,689 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 47) Initializing DWH Heart Beat >2014-09-09 15:47:52,690 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 47) DWH Heart Beat initialized >2014-09-09 15:47:54,584 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-09 15:47:54,601 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-09 15:47:55,079 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:47:55,079 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-09 15:47:55,794 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 15:47:56,565 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488]), log id: 638aa1c2 >2014-09-09 15:47:56,578 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 638aa1c2 >2014-09-09 15:47:56,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: UNASSIGNED not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-09 15:47:56,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: DWH_STOPPED not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: DWH_STARTED not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: DWH_ERROR not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-09 15:47:56,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-09-09 15:47:56,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:47:56,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-09-09 15:47:56,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-09-09 15:47:56,977 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [1da041f2] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:47:56,982 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [1da041f2] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 4ac5f9cf >2014-09-09 15:47:57,023 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [1da041f2] FINISH, SetVdsStatusVDSCommand, log id: 4ac5f9cf >2014-09-09 15:47:57,059 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [1da041f2] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 15:47:57,506 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [1da041f2] Correlation ID: 1da041f2, Job ID: e0d83450-ef71-4dcb-bd0c-25855560fee0, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 15:47:57,580 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [390b0eea] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:47:57,720 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [390b0eea] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488]), log id: 6bee3d2e >2014-09-09 15:47:57,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [390b0eea] FINISH, GetHardwareInfoVDSCommand, log id: 6bee3d2e >2014-09-09 15:47:57,919 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [702723b6] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:47:57,940 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [702723b6] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 22d82eef >2014-09-09 15:47:57,978 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [702723b6] FINISH, SetVdsStatusVDSCommand, log id: 22d82eef >2014-09-09 15:47:58,001 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [702723b6] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 15:47:58,040 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [18fdd1da] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:47:58,075 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [18fdd1da] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 15:47:58,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-6) [18fdd1da] START, DestroyVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vmId=e8715843-38c5-4188-b00e-b9ceadce2714, force=false, secondsToWait=0, gracefully=false, reason=), log id: 42dbc279 >2014-09-09 15:47:58,348 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-6) [18fdd1da] FINISH, DestroyVDSCommand, log id: 42dbc279 >2014-09-09 15:47:58,678 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) [18fdd1da] Initializing Host: rack >2014-09-09 15:47:58,761 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [6357073f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:47:58,765 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) [6357073f] Host 14369bd0-907d-40d7-be1e-bd35af6d1488 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 15:48:05,047 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginAdminUserCommand internal: false. >2014-09-09 15:48:05,087 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-09 15:48:09,021 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-09 15:50:00,106 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) Autorecovering 1 hosts >2014-09-09 15:50:00,106 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) Autorecovering hosts id: 14369bd0-907d-40d7-be1e-bd35af6d1488, name : rack >2014-09-09 15:50:00,119 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:50:00,121 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 15:50:00,122 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 15:50:00,122 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 15:50:00,142 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 10d0f86a >2014-09-09 15:50:00,177 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] FINISH, SetVdsStatusVDSCommand, log id: 10d0f86a >2014-09-09 15:50:00,304 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 15:50:00,336 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Correlation ID: 24b7b64b, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-09 15:50:00,336 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:50:01,899 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-8) [6bb4a5f] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_OPERATIONAL >2014-09-09 15:50:02,202 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, vds=Host[rack,14369bd0-907d-40d7-be1e-bd35af6d1488]), log id: 4c4b22dd >2014-09-09 15:50:02,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GetHardwareInfoVDSCommand, log id: 4c4b22dd >2014-09-09 15:50:02,414 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [46193dd4] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:50:02,446 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [46193dd4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 29c2bda5 >2014-09-09 15:50:02,479 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) [46193dd4] FINISH, SetVdsStatusVDSCommand, log id: 29c2bda5 >2014-09-09 15:50:02,488 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-87) [46193dd4] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 15:50:02,517 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [46193dd4] Correlation ID: 46193dd4, Job ID: 17174584-81e5-4c6e-8f8f-41f6890fd205, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 15:50:02,551 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-87) [a90f6d5] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:50:02,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-87) [a90f6d5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 15:50:02,634 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:50:02,638 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Host 14369bd0-907d-40d7-be1e-bd35af6d1488 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 15:50:20,919 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-10) [6a92276b] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 15:50:20,923 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-10) [6a92276b] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 63087612 >2014-09-09 15:50:20,951 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-10) [6a92276b] FINISH, SetVdsStatusVDSCommand, log id: 63087612 >2014-09-09 15:50:21,148 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-10) [6a92276b] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDS >2014-09-09 15:50:21,165 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-10) [6a92276b] Correlation ID: 6a92276b, Job ID: 28c8875a-05cd-4d63-998b-c03fb8c5a610, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-09 15:50:23,753 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 14369bd0-907d-40d7-be1e-bd35af6d1488 : rack >2014-09-09 15:50:23,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-12) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-09 15:50:23,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-12) Removing vds [14369bd0-907d-40d7-be1e-bd35af6d1488] from the domain in maintenance cache >2014-09-09 15:50:23,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-12) Removing host(s) [14369bd0-907d-40d7-be1e-bd35af6d1488] from hosts unseen domain report cache >2014-09-09 15:51:31,869 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.io.IOException: Connection reset by peer > at sun.nio.ch.FileDispatcherImpl.read0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) [rt.jar:1.7.0_65] > at sun.nio.ch.IOUtil.read(IOUtil.java:197) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:203) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:144) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 15:57:22,257 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-10) [337bc998] Lock Acquired to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 15:57:22,304 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-14) [337bc998] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 14369bd0-907d-40d7-be1e-bd35af6d1488 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-09 15:57:22,409 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-14) [337bc998] START, RemoveVdsVDSCommand( HostId = 14369bd0-907d-40d7-be1e-bd35af6d1488), log id: 6cfabc4 >2014-09-09 15:57:22,409 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-14) [337bc998] vdsManager::disposing >2014-09-09 15:57:22,419 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-14) [337bc998] FINISH, RemoveVdsVDSCommand, log id: 6cfabc4 >2014-09-09 15:57:22,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [337bc998] Correlation ID: 337bc998, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-09 15:57:22,445 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-14) [337bc998] Lock freed to object EngineLock [exclusiveLocks= key: 14369bd0-907d-40d7-be1e-bd35af6d1488 value: VDS >, sharedLocks= ] >2014-09-09 16:24:37,443 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-7) [1f633769] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-09 16:24:37,640 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [207f962e] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:24:37,687 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-7) [207f962e] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-09 16:24:37,691 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] START, RemoveVdsVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77), log id: 3f5d2881 >2014-09-09 16:24:37,691 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] FINISH, RemoveVdsVDSCommand, log id: 3f5d2881 >2014-09-09 16:24:37,694 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] START, AddVdsVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77), log id: 3e80e999 >2014-09-09 16:24:37,695 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] AddVds - entered , starting logic to add VDS 1fc56aac-07f1-49db-8972-b5b9567dee77 >2014-09-09 16:24:37,697 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] AddVds - VDS 1fc56aac-07f1-49db-8972-b5b9567dee77 was added, will try to add it to the resource manager >2014-09-09 16:24:37,697 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [207f962e] Entered VdsManager constructor >2014-09-09 16:24:37,719 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-7) [207f962e] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-09 16:24:37,720 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-7) [207f962e] VDS 1fc56aac-07f1-49db-8972-b5b9567dee77 was added to the Resource Manager >2014-09-09 16:24:37,720 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-7) [207f962e] FINISH, AddVdsVDSCommand, log id: 3e80e999 >2014-09-09 16:24:37,743 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [207f962e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-09 16:24:37,852 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [207f962e] Correlation ID: 1f633769, Job ID: ccf74cb2-cf32-4077-b13a-ace5a1f14e3f, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-09 16:24:37,854 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:24:37,885 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:24:37,893 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Before Installation host 1fc56aac-07f1-49db-8972-b5b9567dee77, rack >2014-09-09 16:24:37,902 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 38d9f719 >2014-09-09 16:24:37,915 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] FINISH, SetVdsStatusVDSCommand, log id: 38d9f719 >2014-09-09 16:24:37,953 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 3f:fe:39:f2:21:7b:9c:a0:53:49:a6:35:b0:6c:4a:ff >2014-09-09 16:24:37,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 3f:fe:39:f2:21:7b:9c:a0:53:49:a6:35:b0:6c:4a:ff. >2014-09-09 16:24:38,032 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-09 16:24:38,045 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-15) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-09 16:24:40,724 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Initializing Host: rack >2014-09-09 16:24:40,913 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-09 16:24:40,935 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-09 16:24:40,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-09 16:24:40,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-09 16:24:40,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-09 16:24:41,027 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-09 16:24:41,027 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-09 16:24:41,044 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-09 16:24:41,044 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-09 16:24:41,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-09 16:24:41,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump unsupported >2014-09-09 16:24:41,086 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump unsupported. >2014-09-09 16:24:41,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140909162438-1rvbah.log' >2014-09-09 16:24:41,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140909162438-1rvbah.log'. >2014-09-09 16:24:41,234 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-09 16:24:41,240 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 16:24:41,357 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-09 16:24:41,383 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-09 16:24:41,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-09 16:24:41,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-09 16:24:41,453 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-09 16:24:41,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-09 16:24:41,469 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-09 16:24:41,480 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-09 16:24:41,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 16:24:41,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 16:24:41,492 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-09 16:24:41,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-09 16:24:48,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 16:24:48,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 16:24:49,120 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-09 16:24:49,133 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-09 16:24:50,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-09 16:24:50,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-09 16:24:50,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-09 16:24:50,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-09 16:24:51,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-09 16:24:51,258 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-09 16:24:51,588 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-09 16:24:51,600 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-09 16:24:53,355 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-09 16:24:53,372 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-09 16:24:53,389 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909162453-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-647ba2dc.log' >2014-09-09 16:24:53,407 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909162453-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-647ba2dc.log'. >2014-09-09 16:24:53,708 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-09 16:24:53,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 647ba2dc, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-09 16:24:53,855 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 16:24:54,010 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Engine managed to communicate with VDSM agent on host rack >2014-09-09 16:24:54,687 WARN [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] The lastClientInterface ovirtmgmt of host rack is not a valid interface for the mangement network. If the interface is a bridge, it should be torn-down manually. >2014-09-09 16:24:54,693 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Host installation failed for host 1fc56aac-07f1-49db-8972-b5b9567dee77, rack.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:206) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: lastClientIface ovirtmgmt is not a valid interface for management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.findNicToSetupManagementNetwork(NetworkConfigurator.java:170) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:74) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:198) [bll.jar:] > ... 70 more > >2014-09-09 16:24:54,702 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 242d4a81 >2014-09-09 16:24:54,727 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] FINISH, SetVdsStatusVDSCommand, log id: 242d4a81 >2014-09-09 16:24:54,770 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Correlation ID: 647ba2dc, Job ID: ccf74cb2-cf32-4077-b13a-ace5a1f14e3f, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. Failed to configure management network on the host. >2014-09-09 16:24:54,782 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-15) [647ba2dc] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:25:00,062 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-90) Autorecovering 1 hosts >2014-09-09 16:25:00,062 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-90) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:25:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:25:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:25:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:25:00,069 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:25:00,074 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 45ec3e9d >2014-09-09 16:25:00,101 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] FINISH, SetVdsStatusVDSCommand, log id: 45ec3e9d >2014-09-09 16:25:00,134 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:25:00,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-90) [18a5c631] Correlation ID: 18a5c631, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-09 16:25:00,146 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:25:01,876 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-95) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 6b0a9c81 >2014-09-09 16:25:01,969 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-95) FINISH, GetHardwareInfoVDSCommand, log id: 6b0a9c81 >2014-09-09 16:25:02,082 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-95) [3feefd64] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,086 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) [3feefd64] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 3ff7b6f9 >2014-09-09 16:25:02,111 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) [3feefd64] FINISH, SetVdsStatusVDSCommand, log id: 3ff7b6f9 >2014-09-09 16:25:02,116 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-95) [3feefd64] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:25:02,139 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [3feefd64] Correlation ID: 3feefd64, Job ID: 4a0b2277-d850-4deb-b2df-b1c10ef2145e, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:25:02,163 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-95) [5f392203] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-95) [5f392203] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 403a6004 >2014-09-09 16:25:02,208 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-95) [5f392203] FINISH, GetHardwareInfoVDSCommand, log id: 403a6004 >2014-09-09 16:25:02,288 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-95) [1d0acec4] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,291 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) [1d0acec4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 6a7e3293 >2014-09-09 16:25:02,302 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) [1d0acec4] FINISH, SetVdsStatusVDSCommand, log id: 6a7e3293 >2014-09-09 16:25:02,336 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-95) [1d0acec4] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:25:02,354 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-95) [6eafe1f4] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,369 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) [6eafe1f4] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:25:02,409 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-95) [13c332f3] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,414 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-95) [24a2c919] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:25:02,416 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) [24a2c919] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:30:00,020 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-69) Autorecovering 1 hosts >2014-09-09 16:30:00,020 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-69) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:30:00,025 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:30:00,025 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:30:00,026 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:30:00,026 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:30:00,035 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 24310dd8 >2014-09-09 16:30:00,070 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] FINISH, SetVdsStatusVDSCommand, log id: 24310dd8 >2014-09-09 16:30:00,137 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:30:00,140 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:30:03,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-77) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 61da0aea >2014-09-09 16:30:03,036 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-77) FINISH, GetHardwareInfoVDSCommand, log id: 61da0aea >2014-09-09 16:30:03,132 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-77) [5460fbba] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:30:03,136 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) [5460fbba] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 2cbced61 >2014-09-09 16:30:03,182 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) [5460fbba] FINISH, SetVdsStatusVDSCommand, log id: 2cbced61 >2014-09-09 16:30:03,190 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-77) [5460fbba] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:30:03,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) [5460fbba] Correlation ID: 5460fbba, Job ID: 47271c9e-6cdd-4f5c-8ea4-631b37faff33, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:30:03,237 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-77) [4275b3b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:30:03,257 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-77) [4275b3b] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:30:03,302 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:30:03,305 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:35:00,030 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-48) Autorecovering 1 hosts >2014-09-09 16:35:00,030 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-48) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:35:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:35:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:35:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:35:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:35:00,045 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 19846f38 >2014-09-09 16:35:00,073 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] FINISH, SetVdsStatusVDSCommand, log id: 19846f38 >2014-09-09 16:35:00,087 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:35:00,090 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-48) [1a5d7bf6] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:35:00,868 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-49) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 6e050268 >2014-09-09 16:35:00,873 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, GetHardwareInfoVDSCommand, log id: 6e050268 >2014-09-09 16:35:00,980 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [69dd153d] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:35:00,984 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [69dd153d] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 95fabe0 >2014-09-09 16:35:01,013 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-49) [69dd153d] FINISH, SetVdsStatusVDSCommand, log id: 95fabe0 >2014-09-09 16:35:01,020 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-49) [69dd153d] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:35:01,042 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [69dd153d] Correlation ID: 69dd153d, Job ID: fd42f06a-cf2e-49b2-8b1d-c141bd88a4be, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:35:01,065 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-49) [7fec9ac8] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:35:01,086 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) [7fec9ac8] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:35:01,133 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-49) [7e37bc27] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:35:01,137 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) [7e37bc27] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:40:00,039 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-25) Autorecovering 1 hosts >2014-09-09 16:40:00,039 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-25) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:40:00,052 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:40:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:40:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:40:00,053 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:40:00,064 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-25) [27472998] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 28b2ecf7 >2014-09-09 16:40:00,112 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-25) [27472998] FINISH, SetVdsStatusVDSCommand, log id: 28b2ecf7 >2014-09-09 16:40:00,154 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:40:00,156 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:40:01,653 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 3d24571c >2014-09-09 16:40:01,659 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-33) FINISH, GetHardwareInfoVDSCommand, log id: 3d24571c >2014-09-09 16:40:01,786 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-33) [51010aba] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:40:01,791 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) [51010aba] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 48e7fb7b >2014-09-09 16:40:01,828 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) [51010aba] FINISH, SetVdsStatusVDSCommand, log id: 48e7fb7b >2014-09-09 16:40:01,836 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-33) [51010aba] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:40:01,856 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [51010aba] Correlation ID: 51010aba, Job ID: 40057208-b2e7-4841-ae54-92c5bf9ecfd0, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:40:01,883 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-33) [61109934] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:40:01,905 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [61109934] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:40:01,967 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-33) [8ff0b8] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:40:01,971 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) [8ff0b8] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:45:00,022 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-8) Autorecovering 1 hosts >2014-09-09 16:45:00,023 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-8) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:45:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:45:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:45:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:45:00,031 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:45:00,044 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 246a5798 >2014-09-09 16:45:00,068 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] FINISH, SetVdsStatusVDSCommand, log id: 246a5798 >2014-09-09 16:45:00,081 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:45:00,084 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [28908be8] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:45:02,501 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 78b8476a >2014-09-09 16:45:02,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GetHardwareInfoVDSCommand, log id: 78b8476a >2014-09-09 16:45:02,659 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-10) [51f90576] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:45:02,662 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [51f90576] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 7a06f81 >2014-09-09 16:45:02,691 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) [51f90576] FINISH, SetVdsStatusVDSCommand, log id: 7a06f81 >2014-09-09 16:45:02,749 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-10) [51f90576] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:45:02,777 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [51f90576] Correlation ID: 51f90576, Job ID: 4ec8fe3f-42fc-44e8-9afe-4205fce8d3d3, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:45:02,801 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-10) [7218fadb] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:45:02,819 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) [7218fadb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:45:02,873 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-10) [5a972ba0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:45:02,876 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) [5a972ba0] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:47:52,248 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-09 16:47:52,587 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) [24a2c919] Attempting to update VMs/Templates Ovf. >2014-09-09 16:50:00,041 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-83) Autorecovering 1 hosts >2014-09-09 16:50:00,041 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-83) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:50:00,046 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:50:00,046 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:50:00,046 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:50:00,046 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:50:00,062 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-83) [b670564] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 92438f6 >2014-09-09 16:50:00,097 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-83) [b670564] FINISH, SetVdsStatusVDSCommand, log id: 92438f6 >2014-09-09 16:50:00,140 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:50:00,144 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:50:00,399 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-89) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 2ebfecda >2014-09-09 16:50:00,404 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-89) FINISH, GetHardwareInfoVDSCommand, log id: 2ebfecda >2014-09-09 16:50:00,500 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-89) [7c825d3] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:50:00,504 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) [7c825d3] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 34604fd6 >2014-09-09 16:50:00,535 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) [7c825d3] FINISH, SetVdsStatusVDSCommand, log id: 34604fd6 >2014-09-09 16:50:00,542 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-89) [7c825d3] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:50:00,563 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) [7c825d3] Correlation ID: 7c825d3, Job ID: 10891a4f-7345-4ea9-832c-c47f89ff1126, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:50:00,584 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-89) [38ee5f8a] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:50:00,600 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) [38ee5f8a] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:50:00,648 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-89) [55b965af] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:50:00,650 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) [55b965af] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 16:55:00,021 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-66) Autorecovering 1 hosts >2014-09-09 16:55:00,021 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-66) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 16:55:00,025 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:55:00,025 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 16:55:00,026 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 16:55:00,026 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 16:55:00,035 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3e4c81e7 >2014-09-09 16:55:00,072 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] FINISH, SetVdsStatusVDSCommand, log id: 3e4c81e7 >2014-09-09 16:55:00,088 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 16:55:00,091 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 16:55:01,168 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 37b6906 >2014-09-09 16:55:01,173 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-72) FINISH, GetHardwareInfoVDSCommand, log id: 37b6906 >2014-09-09 16:55:01,293 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-72) [64385954] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:55:01,299 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) [64385954] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 49285c86 >2014-09-09 16:55:01,339 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) [64385954] FINISH, SetVdsStatusVDSCommand, log id: 49285c86 >2014-09-09 16:55:01,346 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-72) [64385954] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 16:55:01,366 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) [64385954] Correlation ID: 64385954, Job ID: 03478996-4a4d-4ffa-b44f-bc2efc45b340, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 16:55:01,391 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-72) [d9736a9] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:55:01,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-72) [d9736a9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 16:55:01,467 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-72) [3822e820] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 16:55:01,469 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) [3822e820] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 17:00:00,024 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-51) Autorecovering 1 hosts >2014-09-09 17:00:00,024 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-51) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 17:00:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:00:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:00:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 17:00:00,030 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 17:00:00,038 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: b03071 >2014-09-09 17:00:00,063 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] FINISH, SetVdsStatusVDSCommand, log id: b03071 >2014-09-09 17:00:00,076 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 17:00:00,079 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:00:01,989 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-53) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 1675d3ae >2014-09-09 17:00:01,995 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-53) FINISH, GetHardwareInfoVDSCommand, log id: 1675d3ae >2014-09-09 17:00:02,109 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-53) [41fda303] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:00:02,113 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) [41fda303] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 563f2d47 >2014-09-09 17:00:02,148 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) [41fda303] FINISH, SetVdsStatusVDSCommand, log id: 563f2d47 >2014-09-09 17:00:02,154 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-53) [41fda303] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 17:00:02,176 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) [41fda303] Correlation ID: 41fda303, Job ID: 10561b87-ee9c-4f13-b377-485a2585805e, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 17:00:02,197 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-53) [db8b1b9] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:00:02,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) [db8b1b9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 17:00:02,283 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:00:02,286 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 17:05:00,036 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-30) Autorecovering 1 hosts >2014-09-09 17:05:00,036 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-30) Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 17:05:00,042 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:05:00,042 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:05:00,042 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 17:05:00,042 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 17:05:00,052 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6556720e >2014-09-09 17:05:00,077 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] FINISH, SetVdsStatusVDSCommand, log id: 6556720e >2014-09-09 17:05:00,090 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 17:05:00,093 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:05:02,793 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-35) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]), log id: 42eb5942 >2014-09-09 17:05:02,798 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-35) FINISH, GetHardwareInfoVDSCommand, log id: 42eb5942 >2014-09-09 17:05:02,888 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-35) [2ee76c6a] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:05:02,891 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-35) [2ee76c6a] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 6dbb8460 >2014-09-09 17:05:02,917 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-35) [2ee76c6a] FINISH, SetVdsStatusVDSCommand, log id: 6dbb8460 >2014-09-09 17:05:02,923 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-35) [2ee76c6a] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-09 17:05:02,945 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) [2ee76c6a] Correlation ID: 2ee76c6a, Job ID: 54b383cb-fe4a-4faf-a4d5-53f34a6fab1b, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-09 17:05:02,974 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-35) [38973ef1] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:05:03,006 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-35) [38973ef1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-09 17:05:03,056 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:05:03,059 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Host 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-09 17:10:00,030 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-8) [28908be8] Autorecovering 1 hosts >2014-09-09 17:10:00,030 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-8) [28908be8] Autorecovering hosts id: 1fc56aac-07f1-49db-8972-b5b9567dee77, name : rack >2014-09-09 17:10:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:10:00,035 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:10:00,036 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 17:10:00,036 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 17:10:00,045 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 535fb0a6 >2014-09-09 17:10:00,072 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] FINISH, SetVdsStatusVDSCommand, log id: 535fb0a6 >2014-09-09 17:10:00,085 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 17:10:00,088 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-8) [13036fef] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:10:00,510 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:06,107 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:06,108 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:06,109 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-9) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection timed out > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:06,124 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection timed out >2014-09-09 17:11:06,127 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:06,144 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:06,177 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:06,184 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Failed to refresh VDS , vds = 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack, VDS Network Error, continuing. >java.net.ConnectException: Connection timed out >2014-09-09 17:11:09,194 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:12,195 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:12,196 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:12,196 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-36) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:12,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:12,204 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:12,210 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:12,227 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Failed to refresh VDS , vds = 1fc56aac-07f1-49db-8972-b5b9567dee77 : rack, VDS Network Error, continuing. >java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:15,235 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:18,237 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:18,238 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:18,239 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-51) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:18,251 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:18,251 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:18,314 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Server failed to respond, vds_id = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:18,328 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-09 17:11:18,334 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-38) [5c5dd985] ResourceManager::vdsNotResponding entered for Host 1fc56aac-07f1-49db-8972-b5b9567dee77, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:11:18,371 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:18,422 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-38) [3c358b01] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:11:18,426 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-38) [3c358b01] Opening SSH Soft Fencing session on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:11:21,240 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-38) [3c358b01] SSH Soft Fencing command failed on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Failed to get the session. >Stdout: >Stderr: >Stacktrace: org.apache.sshd.common.RuntimeSshException: Failed to get the session.: org.apache.sshd.common.RuntimeSshException: Failed to get the session. > at org.apache.sshd.client.future.DefaultConnectFuture.getSession(DefaultConnectFuture.java:43) > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:294) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:243) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198) [rt.jar:1.7.0_65] > at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213) [rt.jar:1.7.0_65] > at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293) [rt.jar:1.7.0_65] > ... 1 more > >2014-09-09 17:11:21,309 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-38) [483ca656] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 17:11:21,309 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-38) [483ca656] Failed to run Fence script on vds:rack. >2014-09-09 17:11:21,326 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-38) [483ca656] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-09-09 17:11:21,326 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-38) [483ca656] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-09-09 17:11:21,327 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-38) [483ca656] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 17:11:21,391 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:24,239 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:24,240 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:24,240 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-55) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:24,265 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:24,265 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:24,310 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:27,336 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:30,337 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:30,337 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:30,338 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-69) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:30,342 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:30,342 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:30,373 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:33,390 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:36,392 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:36,393 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:36,393 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-84) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:36,396 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:36,396 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:36,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:39,444 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:42,446 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:42,446 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:42,447 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-92) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:42,449 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:42,449 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:42,480 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:45,495 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:48,496 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:48,497 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:48,498 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-11) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:48,503 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:48,503 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:48,528 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:51,543 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:11:54,545 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:54,545 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:54,546 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-17) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:54,549 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-17) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:11:54,550 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-17) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:11:54,582 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:11:57,606 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:07,608 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:07,608 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:07,609 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:07,611 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:07,611 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:07,641 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:10,657 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:13,659 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:13,659 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:13,660 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-59) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:13,663 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:13,663 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:13,696 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:16,714 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:19,716 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:19,716 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:19,717 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-70) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:19,719 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:19,720 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:19,750 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:22,764 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:25,765 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:25,765 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:25,766 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-86) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:25,768 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:25,768 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:25,793 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:28,809 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:31,811 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:31,811 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:31,812 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-95) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:31,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) [24a2c919] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:31,814 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) [24a2c919] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:31,841 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) [24a2c919] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:34,859 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:37,860 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:37,860 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:37,861 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-11) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:37,864 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:37,864 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:37,894 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:40,921 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:43,922 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:43,922 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:43,923 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:43,926 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:43,926 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) [27472998] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:43,959 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) [27472998] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:46,976 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:49,978 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:49,978 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:49,979 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-32) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:49,981 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-32) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:49,982 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:50,017 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-32) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:53,035 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:12:56,036 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:56,036 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:56,037 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-50) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:56,039 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:12:56,040 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:12:56,070 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:12:59,084 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:02,086 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:02,086 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:02,087 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-59) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:02,089 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:13:02,090 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:02,117 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:05,133 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:08,135 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:08,135 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:08,136 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-70) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:08,140 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:13:08,140 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:08,172 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:11,187 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:14,188 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:14,189 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:14,189 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-87) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:14,194 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:13:14,195 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:14,227 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:17,242 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:17,243 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:17,244 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:17,244 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-3) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:17,246 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:13:17,246 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:17,275 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:20,291 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:20,292 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:20,292 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:20,293 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:20,296 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:13:20,296 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:20,333 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:23,359 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:30,361 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:30,361 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:30,361 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:30,365 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:13:30,365 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:13:30,397 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:13:33,414 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:13:33,415 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:295) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:203) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:144) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:35,608 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7b9db3b4 >2014-09-09 17:16:35,608 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) HostName = rack >2014-09-09 17:16:35,608 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:16:35,608 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:16:35,642 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-36) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:16:35,676 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-09 17:16:38,698 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:16:41,700 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:41,700 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:41,701 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-98) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:41,704 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:16:41,704 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:16:41,731 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:44,745 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:16:47,747 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:47,748 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:47,748 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:47,752 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:16:47,753 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:16:47,788 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:50,807 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:16:53,809 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:53,809 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:53,810 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-23) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:53,816 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-23) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:16:53,816 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-23) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:16:53,846 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-23) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:16:56,861 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:14,862 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:14,862 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:14,863 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-40) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:14,865 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:14,866 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:14,919 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:17,949 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:20,950 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:20,950 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:20,951 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-77) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:20,954 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:20,955 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:20,983 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:23,996 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:26,997 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:26,998 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:26,998 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-91) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:27,001 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:27,002 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:27,031 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:30,051 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:33,052 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:33,052 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:33,053 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-100) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:33,056 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:33,057 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:33,094 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:36,124 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:39,125 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:39,125 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:39,126 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-18) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:39,128 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:39,128 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:39,156 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:42,177 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:45,179 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:45,179 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:45,180 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-22) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:45,183 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:45,184 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:45,215 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:48,231 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:51,232 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:51,232 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:51,233 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:51,235 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:51,235 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:51,262 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:54,275 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:17:57,276 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:57,276 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:57,277 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-67) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:17:57,280 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:17:57,280 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:17:57,310 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:00,329 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:03,330 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:03,330 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:03,331 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-71) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:03,339 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:18:03,339 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:18:03,367 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:06,385 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:09,387 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:09,387 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:09,388 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-85) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:09,390 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:18:09,391 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:18:09,426 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:12,443 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:13,445 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:13,446 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:13,446 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-86) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:13,450 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:18:13,450 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:18:13,510 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:16,528 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:16,529 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:16,529 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:16,530 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-95) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:16,533 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-95) [24a2c919] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:18:16,533 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-95) [24a2c919] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:18:16,567 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-95) [24a2c919] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:19,583 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:19,584 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:19,584 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:19,584 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-4) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:19,587 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:18:19,587 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:18:19,624 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:18:22,639 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:18:37,641 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:265) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:474) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:21:39,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7b0b2b6f >2014-09-09 17:21:39,723 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) HostName = rack >2014-09-09 17:21:39,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:21:39,723 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:21:39,741 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:21:39,767 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-09 17:21:42,782 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:21:57,784 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:21:57,784 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:21:57,784 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-2) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:21:57,787 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:21:57,787 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:21:57,828 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:00,855 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:03,856 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:03,856 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:03,857 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-47) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:03,859 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:03,860 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:03,885 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:06,902 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:06,903 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:06,904 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:06,904 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-57) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:06,907 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:06,907 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:06,942 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:09,957 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:09,957 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:09,958 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:09,958 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-67) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:09,960 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:09,961 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:09,991 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:13,017 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:13,018 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:13,019 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:13,019 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-59) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:13,034 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-59) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:13,034 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-59) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:13,065 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:16,079 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:16,080 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:16,080 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:16,081 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-71) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:16,083 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:16,083 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:16,111 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:19,127 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:19,127 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:19,128 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:19,128 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-75) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:19,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:19,132 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:19,166 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:22,188 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:22,189 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:22,189 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:22,189 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:22,192 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:22,192 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) [18a5c631] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:22,221 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) [18a5c631] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:25,239 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:25,240 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:25,240 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:25,240 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-87) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:25,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:25,244 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:25,282 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-87) [53e7c8ba] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:28,302 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:28,302 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:28,303 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:28,303 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-98) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:28,306 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-98) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:28,307 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:28,339 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:31,355 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:31,356 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:31,357 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:31,357 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:31,360 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:31,360 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:31,391 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:34,407 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:34,407 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:34,408 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:34,409 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-18) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:34,417 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:34,417 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:34,448 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:37,464 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:37,465 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:37,465 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:37,466 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-24) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:37,470 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:37,470 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:37,498 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:40,515 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:40,516 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:40,517 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:40,517 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:40,521 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:40,521 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:40,552 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:43,570 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:43,571 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:43,571 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:43,572 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-2) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:43,575 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:43,575 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:43,602 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:46,615 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:46,615 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:46,616 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:46,616 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:46,619 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:46,620 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:46,653 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:49,667 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:49,668 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:49,668 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:49,669 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-47) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:49,671 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:49,671 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:49,697 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:52,712 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:52,713 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:52,713 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:52,714 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-50) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:52,717 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:52,717 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:52,744 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:55,770 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:55,771 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:55,771 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:55,772 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-40) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:55,778 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:55,779 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:55,807 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:58,823 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:22:58,823 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:58,824 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:58,825 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-76) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:22:58,828 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:22:58,828 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:22:58,855 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:01,869 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:01,870 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:01,871 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:01,871 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-77) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:01,873 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:01,874 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:01,905 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:04,918 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:04,918 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:04,919 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:04,919 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-84) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:04,922 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-84) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:04,923 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-84) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:04,951 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:07,965 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:07,966 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:07,966 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:07,967 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-91) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:07,969 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-91) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:07,969 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-91) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:07,996 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-91) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:11,012 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:11,012 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:11,013 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:11,013 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-97) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:11,015 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:11,015 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:11,042 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:14,058 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:14,059 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:14,059 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:14,060 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:14,064 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:14,064 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:14,092 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:17,105 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:17,105 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:17,106 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:17,106 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-11) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:17,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:17,109 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-11) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:17,137 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-11) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:20,150 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:20,151 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:20,151 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:20,152 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-24) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:20,155 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:20,162 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:20,195 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:23,211 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:23,212 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:23,212 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:23,213 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:23,215 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:23,215 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:23,244 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:26,257 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:26,258 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:26,258 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:26,259 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-44) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:26,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:26,261 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:26,314 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:29,328 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:29,329 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:29,330 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:29,330 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-41) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:29,333 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-41) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:29,333 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-41) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:29,363 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-41) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:32,378 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:32,379 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:32,379 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:32,379 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-53) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:32,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:32,382 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:32,411 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:35,425 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:35,426 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:35,426 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:35,427 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-62) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:35,440 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:35,444 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:35,470 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:38,486 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:38,486 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:38,487 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:38,487 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-63) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:38,489 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:38,490 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:38,555 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:41,571 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:41,571 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:41,572 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:41,572 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-66) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:41,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:41,575 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:41,601 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-66) [266d3fbf] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:44,620 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:44,621 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:44,621 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:44,624 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-71) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:44,627 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:44,627 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:44,655 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:47,670 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:47,671 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:47,671 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:47,671 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-77) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:47,674 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:47,674 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:47,738 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) [1d4fb057] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:50,751 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:50,752 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:50,752 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:50,753 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:50,755 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:50,755 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) [18a5c631] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:50,781 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) [18a5c631] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:53,809 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:53,810 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:53,810 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:53,811 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-16) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:53,814 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:53,814 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:53,843 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:56,865 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:56,866 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:56,867 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:56,867 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-7) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:56,870 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:56,870 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:56,899 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:59,917 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:23:59,918 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:59,918 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:59,919 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:23:59,922 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:23:59,922 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:23:59,961 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:02,979 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:02,980 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:02,980 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:02,980 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-30) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:02,984 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:02,984 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) [5b453678] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:03,011 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) [5b453678] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:06,026 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:06,027 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:06,028 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:06,028 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-28) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:06,031 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:06,031 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:06,071 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:09,087 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:09,088 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:09,088 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:09,089 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:09,091 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:09,091 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:09,117 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:12,131 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:12,131 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:12,132 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:12,133 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-39) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:12,136 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:12,136 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:12,167 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:15,182 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:15,183 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:15,183 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:15,183 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-49) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:15,187 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-49) [7e37bc27] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:15,187 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-49) [7e37bc27] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:15,217 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-49) [7e37bc27] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:18,236 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:18,237 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:18,237 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:18,238 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-51) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:18,246 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:18,246 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:18,278 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:21,295 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:21,296 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:21,296 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:21,297 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-68) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:21,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:21,300 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:21,335 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:24,353 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:24,354 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:24,354 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:24,354 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-65) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:24,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:24,357 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:24,397 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:27,418 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:27,418 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:27,419 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:27,419 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-73) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:27,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:27,422 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:27,455 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:30,467 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:30,467 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:30,468 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:30,468 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-75) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:30,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:30,472 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:30,494 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:33,505 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:33,505 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:33,506 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:33,506 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-85) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:33,509 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-85) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:33,510 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-85) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:33,547 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:36,566 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:36,570 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:36,571 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:36,571 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-1) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:36,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:36,576 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:36,607 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:39,625 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:39,626 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:39,626 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:39,627 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-96) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:39,629 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-96) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:39,630 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-96) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:39,667 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:42,686 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:42,687 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:42,687 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:42,688 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-3) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:42,691 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:42,691 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:42,708 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:42,734 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:45,748 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:45,749 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:45,750 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:45,750 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:45,753 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:45,753 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:45,783 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:48,798 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:48,798 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:48,799 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:48,799 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-20) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:48,802 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:48,802 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:48,835 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:51,849 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:51,850 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:51,850 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:51,851 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:51,853 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:51,854 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) [27472998] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:51,883 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) [27472998] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:54,898 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:54,899 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:54,900 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:54,900 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-39) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:54,903 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:54,903 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:54,952 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:57,967 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:24:57,968 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:57,968 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:57,969 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:24:57,971 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:24:57,971 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:24:57,997 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:01,012 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:01,013 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:01,013 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:01,014 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-52) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:01,017 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:01,017 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:01,050 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:04,062 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:04,063 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:04,063 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:04,063 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-62) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:04,065 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:04,065 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:04,086 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:07,097 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:07,098 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:07,098 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:07,098 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-67) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:07,101 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:07,101 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:07,126 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:10,135 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:10,135 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:10,135 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:10,138 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-40) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:10,150 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-40) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:10,150 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-40) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:10,172 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-40) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:13,190 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:13,191 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:13,192 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:13,192 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-80) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:13,195 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:13,195 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:13,225 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:16,246 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:16,247 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:16,248 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:16,248 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-83) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:16,251 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-83) [b670564] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:16,251 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-83) [b670564] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:16,282 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) [b670564] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:19,297 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:19,298 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:19,299 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:19,299 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:19,302 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:19,302 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) [18a5c631] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:19,334 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) [18a5c631] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:22,355 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:22,355 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:22,356 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:22,356 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-86) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:22,360 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-86) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:22,360 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-86) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:22,399 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:25,415 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:25,415 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:25,416 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:25,416 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-3) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:25,419 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:25,420 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:25,464 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:28,480 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:28,480 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:28,481 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:28,481 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-4) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:28,485 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:28,486 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:28,517 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:31,533 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:31,534 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:31,535 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:31,535 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-20) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:31,544 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:31,544 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:31,585 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:34,602 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:34,603 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:34,603 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:34,604 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-9) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:34,607 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:34,607 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:34,640 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:37,654 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:37,655 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:37,655 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:37,656 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-22) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:37,658 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:37,658 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:37,691 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:40,710 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:40,711 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:40,711 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:40,712 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-44) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:40,714 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:40,715 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:40,744 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:43,758 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:43,759 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:43,759 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:43,760 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-42) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:43,763 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-42) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:43,768 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-42) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:43,808 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-42) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:46,825 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:46,825 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:46,826 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:46,827 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-52) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:46,829 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:46,830 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:46,858 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:49,873 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:49,874 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:49,874 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:49,875 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-62) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:49,877 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-62) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:49,878 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-62) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:49,906 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-62) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:52,923 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:52,924 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:52,925 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:52,925 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-72) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:52,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-72) [3822e820] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:52,928 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-72) [3822e820] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:52,975 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-72) [3822e820] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:55,988 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:55,989 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:55,990 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:55,990 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-76) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:55,992 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:55,992 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:56,024 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:59,042 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:25:59,046 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:59,046 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:59,047 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-81) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:25:59,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:25:59,050 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:25:59,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:02,098 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:02,098 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:02,099 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:02,099 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-82) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:02,102 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:02,103 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:02,134 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:05,155 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:05,156 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:05,157 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:05,157 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-94) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:05,160 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:05,161 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:05,192 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:08,208 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:08,208 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:08,209 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:08,210 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-92) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:08,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:08,220 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:08,253 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:11,270 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:11,270 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:11,271 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:11,271 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-3) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:11,273 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-3) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:11,274 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-3) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:11,298 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:14,311 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:14,312 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:14,312 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:14,313 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:14,315 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:14,315 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:14,346 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:17,363 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:17,363 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:17,364 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:17,364 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-30) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:17,367 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:17,367 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) [5b453678] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:17,409 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) [5b453678] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:20,424 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:20,425 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:20,425 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:20,426 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-26) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:20,429 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-26) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:20,429 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:20,460 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:23,474 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:23,475 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:23,476 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:23,476 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-35) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:23,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:23,481 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:23,509 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:26,524 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:26,525 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:26,525 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:26,526 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:26,529 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:26,529 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:26,569 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:29,585 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:29,586 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:29,587 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:29,587 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-46) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:29,590 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-46) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:29,590 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-46) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:29,616 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-46) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:32,632 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:32,632 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:32,633 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:32,634 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:32,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:32,637 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:32,668 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:35,682 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:35,683 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:35,683 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:35,684 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-64) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:35,687 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-64) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:35,687 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-64) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:35,733 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-64) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:38,748 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:38,749 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:38,749 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:38,750 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-55) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:38,752 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-55) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:38,753 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-55) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:38,782 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:41,797 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:41,798 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:41,798 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:41,799 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-73) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:41,802 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:41,802 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:41,830 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:44,844 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:44,845 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:44,845 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:44,846 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-70) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:44,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:44,848 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:44,878 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:47,892 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:47,893 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:47,893 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:47,894 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-82) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:47,896 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-82) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:47,896 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-82) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:47,924 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-82) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:50,940 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:50,941 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:50,941 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:50,942 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-1) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:50,951 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:50,951 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:50,993 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:54,008 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:54,009 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:54,009 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:54,010 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-7) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:54,013 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:54,013 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:54,037 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:57,052 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:26:57,053 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:57,053 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:57,054 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-13) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:26:57,057 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:26:57,057 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:26:57,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:00,107 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:00,108 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:00,108 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:00,109 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-21) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:00,116 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:00,116 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:00,173 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:03,188 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:03,189 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:03,190 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:03,190 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-9) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:03,193 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-9) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:03,193 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-9) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:03,222 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-9) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:06,237 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:06,238 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:06,238 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:06,239 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-19) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:06,242 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-19) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:06,244 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-19) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:06,272 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:09,286 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:09,287 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:09,288 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:09,288 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-35) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:09,291 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:09,291 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:09,324 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-35) [4d22a9e0] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:12,338 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:12,339 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:12,339 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:12,340 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:12,345 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:12,346 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:12,375 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:15,389 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:15,390 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:15,390 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:15,390 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:15,393 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:15,393 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:15,425 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:18,441 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:18,442 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:18,443 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:18,443 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-57) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:18,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:18,450 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:18,480 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:21,500 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:21,501 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:21,502 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:21,502 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-67) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:21,505 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:21,510 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:21,548 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:24,564 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:24,564 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:24,565 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:24,565 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-78) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:24,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-78) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:24,574 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-78) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:24,599 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-78) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:27,613 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:27,614 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:27,615 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:27,615 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-80) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:27,618 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:27,618 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:27,643 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:30,657 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:30,658 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:30,658 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:30,659 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-89) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:30,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-89) [55b965af] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:30,662 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-89) [55b965af] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:30,688 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) [55b965af] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:33,703 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:27:33,704 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:33,704 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:33,705 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-99) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:33,708 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-09 17:27:33,708 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:27:33,742 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:27:36,758 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:28:39,758 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:39,758 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:39,759 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-16) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection timed out > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:39,762 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection timed out >2014-09-09 17:28:39,762 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:28:39,778 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:28:39,803 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection timed out > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection timed out > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:42,821 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:28:45,823 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:45,823 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:45,824 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-22) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:45,826 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-22) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:28:45,827 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-22) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:28:45,866 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:48,883 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:28:51,885 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:51,885 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:51,886 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-43) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:51,889 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:28:51,889 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:28:51,920 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:54,937 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:28:57,939 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:57,939 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:57,940 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-63) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:28:57,942 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-63) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:28:57,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-63) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:28:57,974 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-63) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:00,988 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:03,989 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:03,989 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:03,990 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-74) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:03,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:03,995 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:04,030 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:07,046 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:10,047 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:10,047 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:10,048 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-88) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:10,050 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:10,051 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:10,083 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:13,098 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:16,100 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:16,100 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:16,101 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-94) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:16,103 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:16,104 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:16,136 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:19,154 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:22,155 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:22,155 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:22,156 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-7) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:22,159 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:22,159 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:22,187 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:25,202 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:28,203 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:28,203 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:28,204 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-16) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:28,206 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:28,206 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:28,247 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:31,263 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:49,265 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:49,265 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:49,266 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-31) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:49,271 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-31) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:49,271 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-31) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:49,304 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:52,320 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:29:55,322 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:55,323 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:55,323 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-73) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:55,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:29:55,328 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:29:55,362 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:29:58,381 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:01,382 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:01,382 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:01,383 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:01,386 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:01,386 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) [18a5c631] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:01,416 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) [18a5c631] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:04,434 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:07,436 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:07,436 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:07,437 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-94) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:07,440 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:07,442 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:07,472 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:10,486 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:13,488 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:13,488 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:13,489 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:13,491 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:13,491 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:13,515 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:16,528 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:19,529 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:19,529 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:19,530 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-16) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:19,533 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:19,533 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:19,565 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:22,580 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:25,582 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:25,582 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:25,583 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-44) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:25,585 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-44) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:25,585 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-44) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:25,613 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-44) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:28,628 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:31,629 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:31,630 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:31,630 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-52) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:31,633 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:31,633 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:31,662 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:34,677 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:37,678 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:37,678 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:37,679 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-60) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:37,682 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-60) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:37,682 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-60) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:37,718 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-60) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:40,732 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:43,734 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:43,734 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:43,735 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-71) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:43,738 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-71) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:43,738 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-71) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:43,764 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-71) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:46,779 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:48,780 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:48,781 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:48,782 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-36) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:48,785 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:48,785 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:48,815 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:51,830 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:51,830 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:51,831 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:51,831 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-92) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:51,834 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-92) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:51,834 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-92) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:51,864 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:54,881 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:54,882 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:54,882 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:54,882 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-4) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:54,885 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-4) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:54,885 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-4) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:54,911 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-4) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:57,929 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:30:57,930 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:57,930 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:57,931 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-18) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:30:57,942 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:30:57,942 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:30:57,981 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:00,995 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:00,995 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:00,996 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:00,996 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-24) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:00,999 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:00,999 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:01,027 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:04,043 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:04,044 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:04,044 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:04,045 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-16) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:04,054 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:04,055 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:04,086 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:07,101 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:07,102 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:07,102 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:07,102 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-2) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:07,110 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-2) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:07,111 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-2) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:07,139 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:10,153 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:10,154 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:10,154 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:10,155 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-38) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:10,157 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-38) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:10,157 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-38) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:10,196 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-38) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:13,216 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:13,217 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:13,217 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:13,218 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-43) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:13,221 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:13,222 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:13,249 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:16,263 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:16,264 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:16,264 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:16,265 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-57) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:16,269 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-57) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:16,269 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:16,297 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-57) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:19,310 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:19,311 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:19,312 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:19,312 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-50) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:19,314 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:19,315 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:19,345 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:22,365 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:22,366 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:22,366 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:22,367 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-65) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:22,369 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:22,369 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:22,405 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:25,424 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:25,424 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:25,425 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:25,425 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-80) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:25,428 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:25,428 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:25,455 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:28,474 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:28,478 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:28,478 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:28,478 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-70) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:28,481 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:28,481 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:28,511 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:31,526 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:31,526 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:31,527 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:31,527 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-88) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:31,530 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-88) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:31,530 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-88) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:31,556 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-88) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:34,571 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:34,572 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:34,572 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:34,573 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:34,576 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) [18a5c631] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:34,576 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) [18a5c631] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:34,606 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) [18a5c631] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:37,620 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:37,621 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:37,622 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:37,622 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-97) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:37,624 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-97) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:37,624 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-97) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:37,657 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-97) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:40,672 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:40,673 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:40,673 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:40,674 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-12) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:40,677 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-12) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:40,677 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:40,694 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:40,719 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:43,736 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:43,737 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:43,737 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:43,738 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-18) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:43,741 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-18) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:43,745 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-18) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:43,774 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-18) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:46,789 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:46,793 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:46,794 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:46,795 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-15) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:46,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:46,798 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:46,832 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:49,848 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:49,848 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:49,849 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:49,849 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:49,863 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) [27472998] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:49,863 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) [27472998] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:49,887 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) [27472998] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:52,904 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:52,904 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:52,905 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:52,905 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-39) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:52,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-39) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:52,908 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-39) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:52,939 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-39) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:55,954 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:55,954 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:55,955 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:55,955 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-53) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:55,958 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:55,959 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:55,987 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) [5733dd3e] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:59,003 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:31:59,004 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:59,031 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:59,032 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:31:59,034 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:31:59,034 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:31:59,061 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:02,075 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:32:02,076 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:02,076 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:02,076 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-68) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:02,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:32:02,079 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-68) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:32:02,107 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:05,035 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-33) [64ef3315] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:32:05,122 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:32:05,122 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:05,123 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:05,125 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-65) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:05,128 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-65) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, vds=Host[rack,1fc56aac-07f1-49db-8972-b5b9567dee77]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-09 17:32:05,128 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-65) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:32:05,175 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-65) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:32:05,180 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-33) [64ef3315] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDS >2014-09-09 17:32:05,194 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-33) [64ef3315] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7fe96046 >2014-09-09 17:32:05,217 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-33) [64ef3315] FINISH, SetVdsStatusVDSCommand, log id: 7fe96046 >2014-09-09 17:32:05,229 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-33) [64ef3315] Correlation ID: 64ef3315, Job ID: bf9a2a2d-aa3e-4ec7-afac-34668f6e3c4e, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-09 17:32:17,779 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [2d507abd] Lock Acquired to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:32:17,795 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-35) [2d507abd] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 1fc56aac-07f1-49db-8972-b5b9567dee77 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-09 17:32:17,826 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-35) [2d507abd] START, RemoveVdsVDSCommand( HostId = 1fc56aac-07f1-49db-8972-b5b9567dee77), log id: 668ff23e >2014-09-09 17:32:17,827 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-35) [2d507abd] vdsManager::disposing >2014-09-09 17:32:17,828 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-35) [2d507abd] FINISH, RemoveVdsVDSCommand, log id: 668ff23e >2014-09-09 17:32:17,836 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [2d507abd] Correlation ID: 2d507abd, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-09 17:32:17,837 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-35) [2d507abd] Lock freed to object EngineLock [exclusiveLocks= key: 1fc56aac-07f1-49db-8972-b5b9567dee77 value: VDS >, sharedLocks= ] >2014-09-09 17:32:33,607 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-8) [d4f8ee6] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' >2014-09-09 17:32:33,637 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-8) [d4f8ee6] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-09 17:32:33,712 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-8) [139b591] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-09 17:32:33,727 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-8) [139b591] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-09 17:32:33,730 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] START, RemoveVdsVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: 26d724be >2014-09-09 17:32:33,731 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] FINISH, RemoveVdsVDSCommand, log id: 26d724be >2014-09-09 17:32:33,733 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] START, AddVdsVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: d2b5bbd >2014-09-09 17:32:33,733 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] AddVds - entered , starting logic to add VDS 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 >2014-09-09 17:32:33,736 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] AddVds - VDS 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 was added, will try to add it to the resource manager >2014-09-09 17:32:33,736 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-8) [139b591] Entered VdsManager constructor >2014-09-09 17:32:33,749 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-8) [139b591] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-09 17:32:33,750 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-8) [139b591] VDS 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 was added to the Resource Manager >2014-09-09 17:32:33,751 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [139b591] FINISH, AddVdsVDSCommand, log id: d2b5bbd >2014-09-09 17:32:33,766 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [139b591] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-09 17:32:33,804 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-09 17:32:33,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [139b591] Correlation ID: d4f8ee6, Job ID: 28ad829d-a61d-4309-a303-4bf2b7ff5d54, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-09 17:32:33,823 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-09 17:32:33,823 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Before Installation host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, rack >2014-09-09 17:32:33,828 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [6093dd15] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 182d4fcc >2014-09-09 17:32:33,838 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [6093dd15] FINISH, SetVdsStatusVDSCommand, log id: 182d4fcc >2014-09-09 17:32:33,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1 >2014-09-09 17:32:33,889 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1. >2014-09-09 17:32:33,954 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-09 17:32:33,967 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-36) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-09 17:32:36,753 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Initializing Host: rack >2014-09-09 17:32:55,661 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-09 17:32:55,673 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-09 17:32:56,654 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libselinux-python-2.0.94-5.3.el6_4.1.x86_64 >2014-09-09 17:32:56,668 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libselinux-python-2.0.94-5.3.el6_4.1.x86_64. >2014-09-09 17:32:56,668 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-09 17:32:56,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-09 17:32:56,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-09 17:32:56,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-09 17:32:56,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-09 17:32:56,811 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-09 17:32:57,295 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 1/1: libselinux-python-2.0.94-5.3.el6_4.1.x86_64 >2014-09-09 17:32:57,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 1/1: libselinux-python-2.0.94-5.3.el6_4.1.x86_64. >2014-09-09 17:33:02,641 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.3.el6_4.1 - u >2014-09-09 17:33:02,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.3.el6_4.1 - u. >2014-09-09 17:33:09,574 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-09 17:33:09,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-09 17:33:09,600 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-09 17:33:09,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-09 17:33:09,697 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-09 17:33:09,738 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-09 17:33:22,424 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-09 17:33:22,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-09 17:33:22,887 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:dmidecode-2.12-5.el6_5.x86_64 >2014-09-09 17:33:22,903 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:dmidecode-2.12-5.el6_5.x86_64. >2014-09-09 17:33:22,904 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: iproute-2.6.32-32.el6_5.x86_64 >2014-09-09 17:33:22,918 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: iproute-2.6.32-32.el6_5.x86_64. >2014-09-09 17:33:22,918 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-09 17:33:22,929 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-09 17:33:22,929 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-09 17:33:22,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-09 17:33:23,196 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-09 17:33:23,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-09 17:33:23,452 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 1/4: 1:dmidecode-2.12-5.el6_5.x86_64 >2014-09-09 17:33:23,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 1/4: 1:dmidecode-2.12-5.el6_5.x86_64. >2014-09-09 17:33:23,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 2/4: iproute-2.6.32-32.el6_5.x86_64 >2014-09-09 17:33:23,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 2/4: iproute-2.6.32-32.el6_5.x86_64. >2014-09-09 17:33:24,610 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 3/4: dmidecode >2014-09-09 17:33:24,624 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 3/4: dmidecode. >2014-09-09 17:33:24,942 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 4/4: iproute >2014-09-09 17:33:24,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 4/4: iproute. >2014-09-09 17:33:31,416 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/4: iproute.x86_64 0:2.6.32-32.el6_5 - u >2014-09-09 17:33:31,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/4: iproute.x86_64 0:2.6.32-32.el6_5 - u. >2014-09-09 17:33:31,498 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 2/4: dmidecode.x86_64 1:2.12-5.el6_5 - u >2014-09-09 17:33:31,513 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 2/4: dmidecode.x86_64 1:2.12-5.el6_5 - u. >2014-09-09 17:33:31,583 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 3/4: dmidecode.x86_64 1:2.11-2.el6 - ud >2014-09-09 17:33:31,597 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 3/4: dmidecode.x86_64 1:2.11-2.el6 - ud. >2014-09-09 17:33:31,597 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 4/4: iproute.x86_64 0:2.6.32-31.el6 - ud >2014-09-09 17:33:31,610 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 4/4: iproute.x86_64 0:2.6.32-31.el6 - ud. >2014-09-09 17:33:31,854 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-09 17:33:31,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-09 17:33:31,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-09 17:33:31,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-09 17:33:32,321 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-09 17:33:32,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-09 17:33:32,374 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140909173234-j5oav3.log' >2014-09-09 17:33:32,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140909173234-j5oav3.log'. >2014-09-09 17:33:32,497 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-09 17:33:32,527 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:33:32,680 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-09 17:33:32,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-09 17:33:32,730 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-09 17:33:32,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-09 17:33:32,795 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-09 17:33:32,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-09 17:33:33,035 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-09 17:33:33,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-09 17:33:33,127 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 17:33:33,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 17:33:33,141 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-09 17:33:33,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-09 17:34:03,253 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Downloading: rhel65-brq/filelists_db (0%) >2014-09-09 17:34:03,274 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Downloading: rhel65-brq/filelists_db (0%). >2014-09-09 17:34:13,085 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-09 17:34:13,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-09 17:34:13,895 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: audit-2.2-4.el6_5.x86_64 >2014-09-09 17:34:13,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: audit-2.2-4.el6_5.x86_64. >2014-09-09 17:34:14,547 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: audit-libs-2.2-4.el6_5.x86_64 >2014-09-09 17:34:14,562 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: audit-libs-2.2-4.el6_5.x86_64. >2014-09-09 17:34:15,285 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: audit-libs-python-2.2-4.el6_5.x86_64 >2014-09-09 17:34:15,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: audit-libs-python-2.2-4.el6_5.x86_64. >2014-09-09 17:34:15,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: augeas-1.0.0-5.el6_5.1.x86_64 >2014-09-09 17:34:15,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: augeas-1.0.0-5.el6_5.1.x86_64. >2014-09-09 17:34:16,878 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: augeas-libs-1.0.0-5.el6_5.1.x86_64 >2014-09-09 17:34:16,890 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: augeas-libs-1.0.0-5.el6_5.1.x86_64. >2014-09-09 17:34:17,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64 >2014-09-09 17:34:17,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64. >2014-09-09 17:34:18,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: celt051-0.5.1.3-0.el6.x86_64 >2014-09-09 17:34:18,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: celt051-0.5.1.3-0.el6.x86_64. >2014-09-09 17:34:20,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: cvs-1.11.23-16.el6.x86_64 >2014-09-09 17:34:20,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: cvs-1.11.23-16.el6.x86_64. >2014-09-09 17:34:21,488 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 >2014-09-09 17:34:21,499 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64. >2014-09-09 17:34:22,501 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:34:22,513 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:34:23,354 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:34:23,365 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:34:24,288 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: dnsmasq-2.48-13.el6.x86_64 >2014-09-09 17:34:24,299 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: dnsmasq-2.48-13.el6.x86_64. >2014-09-09 17:34:25,078 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ebtables-2.0.9-6.el6.x86_64 >2014-09-09 17:34:25,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: ebtables-2.0.9-6.el6.x86_64. >2014-09-09 17:34:26,113 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: febootstrap-supermin-helper-3.21-4.el6.x86_64 >2014-09-09 17:34:26,124 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: febootstrap-supermin-helper-3.21-4.el6.x86_64. >2014-09-09 17:34:26,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-3.1.5-35.el6_5.4.x86_64 >2014-09-09 17:34:26,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: fence-agents-3.1.5-35.el6_5.4.x86_64. >2014-09-09 17:34:27,963 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: flac-1.2.1-6.1.el6.x86_64 >2014-09-09 17:34:27,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: flac-1.2.1-6.1.el6.x86_64. >2014-09-09 17:34:28,705 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fuse-2.8.3-4.el6.x86_64 >2014-09-09 17:34:28,717 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: fuse-2.8.3-4.el6.x86_64. >2014-09-09 17:34:29,328 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fuse-libs-2.8.3-4.el6.x86_64 >2014-09-09 17:34:29,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: fuse-libs-2.8.3-4.el6.x86_64. >2014-09-09 17:34:30,335 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: genisoimage-1.1.9-12.el6.x86_64 >2014-09-09 17:34:30,349 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: genisoimage-1.1.9-12.el6.x86_64. >2014-09-09 17:34:32,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gettext-0.17-16.el6.x86_64 >2014-09-09 17:34:32,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: gettext-0.17-16.el6.x86_64. >2014-09-09 17:34:37,184 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ghostscript-8.70-19.el6.x86_64 >2014-09-09 17:34:37,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: ghostscript-8.70-19.el6.x86_64. >2014-09-09 17:34:39,880 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ghostscript-fonts-5.50-23.1.el6.noarch >2014-09-09 17:34:39,895 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: ghostscript-fonts-5.50-23.1.el6.noarch. >2014-09-09 17:34:42,364 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-3.5.2-1.el6.x86_64 >2014-09-09 17:34:42,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: glusterfs-3.5.2-1.el6.x86_64. >2014-09-09 17:34:42,953 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-api-3.5.2-1.el6.x86_64 >2014-09-09 17:34:42,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: glusterfs-api-3.5.2-1.el6.x86_64. >2014-09-09 17:34:43,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-libs-3.5.2-1.el6.x86_64 >2014-09-09 17:34:44,003 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: glusterfs-libs-3.5.2-1.el6.x86_64. >2014-09-09 17:34:44,278 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gnutls-2.8.5-14.el6_5.x86_64 >2014-09-09 17:34:44,296 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: gnutls-2.8.5-14.el6_5.x86_64. >2014-09-09 17:34:44,765 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gnutls-utils-2.8.5-14.el6_5.x86_64 >2014-09-09 17:34:44,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: gnutls-utils-2.8.5-14.el6_5.x86_64. >2014-09-09 17:34:45,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gpxe-roms-qemu-0.9.7-6.10.el6.noarch >2014-09-09 17:34:45,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: gpxe-roms-qemu-0.9.7-6.10.el6.noarch. >2014-09-09 17:34:46,826 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: hivex-1.3.3-4.2.el6.x86_64 >2014-09-09 17:34:46,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: hivex-1.3.3-4.2.el6.x86_64. >2014-09-09 17:34:46,842 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ioprocess-0.12.0-2.el6.x86_64 >2014-09-09 17:34:46,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: ioprocess-0.12.0-2.el6.x86_64. >2014-09-09 17:34:48,078 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ipmitool-1.8.11-16.el6.x86_64 >2014-09-09 17:34:48,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: ipmitool-1.8.11-16.el6.x86_64. >2014-09-09 17:34:48,721 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: keyutils-1.4-4.el6.x86_64 >2014-09-09 17:34:48,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: keyutils-1.4-4.el6.x86_64. >2014-09-09 17:34:48,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: kpartx-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:34:48,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: kpartx-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:34:49,477 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXfont-1.4.5-3.el6_5.x86_64 >2014-09-09 17:34:49,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libXfont-1.4.5-3.el6_5.x86_64. >2014-09-09 17:34:50,193 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXtst-1.2.1-2.el6.x86_64 >2014-09-09 17:34:50,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libXtst-1.2.1-2.el6.x86_64. >2014-09-09 17:34:51,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libasyncns-0.8-1.1.el6.x86_64 >2014-09-09 17:34:51,063 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libasyncns-0.8-1.1.el6.x86_64. >2014-09-09 17:34:51,964 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libcgroup-0.40.rc1-5.el6_5.1.x86_64 >2014-09-09 17:34:51,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libcgroup-0.40.rc1-5.el6_5.1.x86_64. >2014-09-09 17:34:52,822 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libconfig-1.3.2-1.1.el6.x86_64 >2014-09-09 17:34:52,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libconfig-1.3.2-1.1.el6.x86_64. >2014-09-09 17:34:53,590 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libevent-1.4.13-4.el6.x86_64 >2014-09-09 17:34:53,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libevent-1.4.13-4.el6.x86_64. >2014-09-09 17:34:54,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libfontenc-1.0.5-2.el6.x86_64 >2014-09-09 17:34:54,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libfontenc-1.0.5-2.el6.x86_64. >2014-09-09 17:34:54,977 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libgomp-4.4.7-4.el6.x86_64 >2014-09-09 17:34:54,989 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libgomp-4.4.7-4.el6.x86_64. >2014-09-09 17:34:55,707 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libgssglue-0.1-11.el6.x86_64 >2014-09-09 17:34:55,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libgssglue-0.1-11.el6.x86_64. >2014-09-09 17:34:58,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libguestfs-1.20.11-2.el6.x86_64 >2014-09-09 17:34:58,702 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:libguestfs-1.20.11-2.el6.x86_64. >2014-09-09 17:35:01,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64 >2014-09-09 17:35:01,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64. >2014-09-09 17:35:02,308 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:libogg-1.1.4-2.1.el6.x86_64 >2014-09-09 17:35:02,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:libogg-1.1.4-2.1.el6.x86_64. >2014-09-09 17:35:03,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libsemanage-python-2.0.43-4.2.el6.x86_64 >2014-09-09 17:35:03,120 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libsemanage-python-2.0.43-4.2.el6.x86_64. >2014-09-09 17:35:04,045 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libsndfile-1.0.20-5.el6.x86_64 >2014-09-09 17:35:04,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libsndfile-1.0.20-5.el6.x86_64. >2014-09-09 17:35:04,775 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libtirpc-0.2.1-6.el6_5.2.x86_64 >2014-09-09 17:35:04,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libtirpc-0.2.1-6.el6_5.2.x86_64. >2014-09-09 17:35:07,922 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:35:07,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libvirt-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:35:11,835 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-client-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:35:11,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libvirt-client-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:35:12,619 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:35:12,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:35:14,239 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-python-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:35:14,251 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libvirt-python-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:35:15,156 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libvorbis-1.2.3-4.el6_2.1.x86_64 >2014-09-09 17:35:15,183 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:libvorbis-1.2.3-4.el6_2.1.x86_64. >2014-09-09 17:35:16,117 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lm_sensors-libs-3.1.1-17.el6.x86_64 >2014-09-09 17:35:16,129 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: lm_sensors-libs-3.1.1-17.el6.x86_64. >2014-09-09 17:35:16,150 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lzop-1.02-0.9.rc1.el6.x86_64 >2014-09-09 17:35:16,163 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: lzop-1.02-0.9.rc1.el6.x86_64. >2014-09-09 17:35:16,193 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: mom-0.4.1-2.el6.noarch >2014-09-09 17:35:16,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: mom-0.4.1-2.el6.noarch. >2014-09-09 17:35:17,036 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: nc-1.84-22.el6.x86_64 >2014-09-09 17:35:17,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: nc-1.84-22.el6.x86_64. >2014-09-09 17:35:18,578 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64 >2014-09-09 17:35:18,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64. >2014-09-09 17:35:19,529 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64 >2014-09-09 17:35:19,541 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64. >2014-09-09 17:35:20,259 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netcf-libs-0.1.9-4.el6_5.2.x86_64 >2014-09-09 17:35:20,270 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: netcf-libs-0.1.9-4.el6_5.2.x86_64. >2014-09-09 17:35:22,459 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netpbm-10.47.05-11.el6.x86_64 >2014-09-09 17:35:22,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: netpbm-10.47.05-11.el6.x86_64. >2014-09-09 17:35:25,967 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netpbm-progs-10.47.05-11.el6.x86_64 >2014-09-09 17:35:25,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: netpbm-progs-10.47.05-11.el6.x86_64. >2014-09-09 17:35:27,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64 >2014-09-09 17:35:27,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64. >2014-09-09 17:35:27,869 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: nfs-utils-lib-1.1.5-6.el6_5.x86_64 >2014-09-09 17:35:27,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: nfs-utils-lib-1.1.5-6.el6_5.x86_64. >2014-09-09 17:35:28,652 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: numad-0.5-10.20140620git.el6_5.x86_64 >2014-09-09 17:35:28,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: numad-0.5-10.20140620git.el6_5.x86_64. >2014-09-09 17:35:29,559 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Net-Telnet-3.03-11.el6.noarch >2014-09-09 17:35:29,571 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: perl-Net-Telnet-3.03-11.el6.noarch. >2014-09-09 17:35:30,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-hivex-1.3.3-4.2.el6.x86_64 >2014-09-09 17:35:30,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: perl-hivex-1.3.3-4.2.el6.x86_64. >2014-09-09 17:35:31,124 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pexpect-2.3-6.el6.noarch >2014-09-09 17:35:31,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: pexpect-2.3-6.el6.noarch. >2014-09-09 17:35:32,397 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: policycoreutils-python-2.0.83-19.39.el6.x86_64 >2014-09-09 17:35:32,413 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: policycoreutils-python-2.0.83-19.39.el6.x86_64. >2014-09-09 17:35:33,725 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pulseaudio-libs-0.9.21-14.el6_3.x86_64 >2014-09-09 17:35:33,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: pulseaudio-libs-0.9.21-14.el6_3.x86_64. >2014-09-09 17:35:34,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pyparted-3.4-4.el6.x86_64 >2014-09-09 17:35:34,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: pyparted-3.4-4.el6.x86_64. >2014-09-09 17:35:34,946 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-argparse-1.2.1-2.1.el6.noarch >2014-09-09 17:35:34,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-argparse-1.2.1-2.1.el6.noarch. >2014-09-09 17:35:34,963 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-cpopen-1.3-4.el6_5.x86_64 >2014-09-09 17:35:34,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-cpopen-1.3-4.el6_5.x86_64. >2014-09-09 17:35:35,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-decorator-3.0.1-3.1.el6.noarch >2014-09-09 17:35:35,518 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-decorator-3.0.1-3.1.el6.noarch. >2014-09-09 17:35:35,787 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-inotify-0.9.1-1.el6ev.noarch >2014-09-09 17:35:35,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-inotify-0.9.1-1.el6ev.noarch. >2014-09-09 17:35:35,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-ioprocess-0.12.0-2.el6.noarch >2014-09-09 17:35:35,811 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-ioprocess-0.12.0-2.el6.noarch. >2014-09-09 17:35:37,556 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-netaddr-0.7.5-4.el6.noarch >2014-09-09 17:35:37,567 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-netaddr-0.7.5-4.el6.noarch. >2014-09-09 17:35:37,720 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-ordereddict-1.1-3.el6ev.noarch >2014-09-09 17:35:37,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-ordereddict-1.1-3.el6ev.noarch. >2014-09-09 17:35:37,820 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-pthreading-0.1.3-3.el6ev.noarch >2014-09-09 17:35:37,831 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-pthreading-0.1.3-3.el6ev.noarch. >2014-09-09 17:35:38,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-suds-0.4.1-3.el6.noarch >2014-09-09 17:35:38,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: python-suds-0.4.1-3.el6.noarch. >2014-09-09 17:35:39,929 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:35:39,942 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:35:40,570 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:35:40,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:35:40,928 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:35:40,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:35:41,611 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: radvd-1.6-1.el6.x86_64 >2014-09-09 17:35:41,623 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: radvd-1.6-1.el6.x86_64. >2014-09-09 17:35:42,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: rpcbind-0.2.0-11.el6.x86_64 >2014-09-09 17:35:42,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: rpcbind-0.2.0-11.el6.x86_64. >2014-09-09 17:35:43,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-2.8-1.el6.x86_64 >2014-09-09 17:35:43,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: sanlock-2.8-1.el6.x86_64. >2014-09-09 17:35:43,816 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-lib-2.8-1.el6.x86_64 >2014-09-09 17:35:43,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: sanlock-lib-2.8-1.el6.x86_64. >2014-09-09 17:35:44,569 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-python-2.8-1.el6.x86_64 >2014-09-09 17:35:44,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: sanlock-python-2.8-1.el6.x86_64. >2014-09-09 17:35:45,178 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: scrub-2.2-1.el6.x86_64 >2014-09-09 17:35:45,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: scrub-2.2-1.el6.x86_64. >2014-09-09 17:35:46,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: seabios-0.6.1.2-28.el6.x86_64 >2014-09-09 17:35:46,045 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: seabios-0.6.1.2-28.el6.x86_64. >2014-09-09 17:35:47,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: setools-libs-3.3.7-4.el6.x86_64 >2014-09-09 17:35:47,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: setools-libs-3.3.7-4.el6.x86_64. >2014-09-09 17:35:48,096 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: setools-libs-python-3.3.7-4.el6.x86_64 >2014-09-09 17:35:48,111 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: setools-libs-python-3.3.7-4.el6.x86_64. >2014-09-09 17:35:49,552 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sg3_utils-1.28-5.el6.x86_64 >2014-09-09 17:35:49,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: sg3_utils-1.28-5.el6.x86_64. >2014-09-09 17:35:50,080 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sgabios-bin-0-0.3.20110621svn.el6.noarch >2014-09-09 17:35:50,094 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: sgabios-bin-0-0.3.20110621svn.el6.noarch. >2014-09-09 17:35:50,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: spice-server-0.12.4-6.el6_5.2.x86_64 >2014-09-09 17:35:50,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: spice-server-0.12.4-6.el6_5.2.x86_64. >2014-09-09 17:35:51,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:telnet-0.17-47.el6_3.1.x86_64 >2014-09-09 17:35:51,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:telnet-0.17-47.el6_3.1.x86_64. >2014-09-09 17:35:52,347 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: tree-1.5.3-2.el6.x86_64 >2014-09-09 17:35:52,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: tree-1.5.3-2.el6.x86_64. >2014-09-09 17:35:55,039 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: urw-fonts-2.4-10.el6.noarch >2014-09-09 17:35:55,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: urw-fonts-2.4-10.el6.noarch. >2014-09-09 17:35:55,685 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: usbredir-0.5.1-1.el6.x86_64 >2014-09-09 17:35:55,699 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: usbredir-0.5.1-1.el6.x86_64. >2014-09-09 17:35:55,716 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-4.16.3-2.el6.x86_64 >2014-09-09 17:35:55,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-4.16.3-2.el6.x86_64. >2014-09-09 17:35:55,731 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-cli-4.16.3-2.el6.noarch >2014-09-09 17:35:55,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-cli-4.16.3-2.el6.noarch. >2014-09-09 17:35:55,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-jsonrpc-4.16.3-2.el6.noarch >2014-09-09 17:35:55,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-jsonrpc-4.16.3-2.el6.noarch. >2014-09-09 17:35:55,757 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-python-4.16.3-2.el6.x86_64 >2014-09-09 17:35:55,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-python-4.16.3-2.el6.x86_64. >2014-09-09 17:35:55,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-2.el6.noarch >2014-09-09 17:35:55,781 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-2.el6.noarch. >2014-09-09 17:35:55,782 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-xmlrpc-4.16.3-2.el6.noarch >2014-09-09 17:35:55,794 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-xmlrpc-4.16.3-2.el6.noarch. >2014-09-09 17:35:55,795 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-yajsonrpc-4.16.3-2.el6.noarch >2014-09-09 17:35:55,806 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vdsm-yajsonrpc-4.16.3-2.el6.noarch. >2014-09-09 17:35:56,452 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vgabios-0.6b-3.7.el6.noarch >2014-09-09 17:35:56,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: vgabios-0.6b-3.7.el6.noarch. >2014-09-09 17:35:57,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:xorg-x11-font-utils-7.2-11.el6.x86_64 >2014-09-09 17:35:57,293 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 1:xorg-x11-font-utils-7.2-11.el6.x86_64. >2014-09-09 17:35:57,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: yajl-1.0.7-3.el6.x86_64 >2014-09-09 17:35:57,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: yajl-1.0.7-3.el6.x86_64. >2014-09-09 17:35:57,988 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-09 17:35:58,002 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-09 17:35:58,003 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Confirm use of GPG Key userid=Red Hat, Inc. (release key 2) <security@redhat.com> hexkeyid=FD431D51 >2014-09-09 17:35:58,017 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Host rack installation in progress . Confirm use of GPG Key userid=Red Hat, Inc. (release key 2) <security@redhat.com> hexkeyid=FD431D51. >2014-09-09 17:35:58,172 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Confirm use of GPG Key userid=Red Hat, Inc. (auxiliary key) <security@redhat.com> hexkeyid=2FA658E0 >2014-09-09 17:35:58,186 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Host rack installation in progress . Confirm use of GPG Key userid=Red Hat, Inc. (auxiliary key) <security@redhat.com> hexkeyid=2FA658E0. >2014-09-09 17:35:58,309 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-09 17:35:58,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-09 17:35:58,971 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-09 17:35:58,987 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-09 17:35:59,423 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 1/113: audit-libs-2.2-4.el6_5.x86_64 >2014-09-09 17:35:59,438 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 1/113: audit-libs-2.2-4.el6_5.x86_64. >2014-09-09 17:35:59,834 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 2/113: yajl-1.0.7-3.el6.x86_64 >2014-09-09 17:35:59,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 2/113: yajl-1.0.7-3.el6.x86_64. >2014-09-09 17:36:00,240 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 3/113: gnutls-2.8.5-14.el6_5.x86_64 >2014-09-09 17:36:00,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 3/113: gnutls-2.8.5-14.el6_5.x86_64. >2014-09-09 17:36:00,857 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 4/113: libgssglue-0.1-11.el6.x86_64 >2014-09-09 17:36:00,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 4/113: libgssglue-0.1-11.el6.x86_64. >2014-09-09 17:36:01,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 5/113: glusterfs-libs-3.5.2-1.el6.x86_64 >2014-09-09 17:36:01,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 5/113: glusterfs-libs-3.5.2-1.el6.x86_64. >2014-09-09 17:36:01,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 6/113: 2:libogg-1.1.4-2.1.el6.x86_64 >2014-09-09 17:36:01,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 6/113: 2:libogg-1.1.4-2.1.el6.x86_64. >2014-09-09 17:36:02,191 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 7/113: augeas-libs-1.0.0-5.el6_5.1.x86_64 >2014-09-09 17:36:02,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 7/113: augeas-libs-1.0.0-5.el6_5.1.x86_64. >2014-09-09 17:36:02,890 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 8/113: sanlock-lib-2.8-1.el6.x86_64 >2014-09-09 17:36:02,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 8/113: sanlock-lib-2.8-1.el6.x86_64. >2014-09-09 17:36:03,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 9/113: libcgroup-0.40.rc1-5.el6_5.1.x86_64 >2014-09-09 17:36:03,572 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 9/113: libcgroup-0.40.rc1-5.el6_5.1.x86_64. >2014-09-09 17:36:04,166 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 10/113: python-cpopen-1.3-4.el6_5.x86_64 >2014-09-09 17:36:04,181 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 10/113: python-cpopen-1.3-4.el6_5.x86_64. >2014-09-09 17:36:05,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 11/113: sanlock-2.8-1.el6.x86_64 >2014-09-09 17:36:05,117 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 11/113: sanlock-2.8-1.el6.x86_64. >2014-09-09 17:36:05,515 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 12/113: libtirpc-0.2.1-6.el6_5.2.x86_64 >2014-09-09 17:36:05,531 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 12/113: libtirpc-0.2.1-6.el6_5.2.x86_64. >2014-09-09 17:36:05,932 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 13/113: perl-hivex-1.3.3-4.2.el6.x86_64 >2014-09-09 17:36:05,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 13/113: perl-hivex-1.3.3-4.2.el6.x86_64. >2014-09-09 17:36:06,448 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 14/113: hivex-1.3.3-4.2.el6.x86_64 >2014-09-09 17:36:06,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 14/113: hivex-1.3.3-4.2.el6.x86_64. >2014-09-09 17:36:06,938 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 15/113: lzop-1.02-0.9.rc1.el6.x86_64 >2014-09-09 17:36:06,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 15/113: lzop-1.02-0.9.rc1.el6.x86_64. >2014-09-09 17:36:07,331 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 16/113: libfontenc-1.0.5-2.el6.x86_64 >2014-09-09 17:36:07,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 16/113: libfontenc-1.0.5-2.el6.x86_64. >2014-09-09 17:36:07,748 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 17/113: fuse-libs-2.8.3-4.el6.x86_64 >2014-09-09 17:36:07,759 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 17/113: fuse-libs-2.8.3-4.el6.x86_64. >2014-09-09 17:36:08,156 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 18/113: usbredir-0.5.1-1.el6.x86_64 >2014-09-09 17:36:08,168 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 18/113: usbredir-0.5.1-1.el6.x86_64. >2014-09-09 17:36:08,573 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 19/113: vdsm-python-zombiereaper-4.16.3-2.el6.noarch >2014-09-09 17:36:08,587 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 19/113: vdsm-python-zombiereaper-4.16.3-2.el6.noarch. >2014-09-09 17:36:08,848 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 20/113: libXfont-1.4.5-3.el6_5.x86_64 >2014-09-09 17:36:08,861 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 20/113: libXfont-1.4.5-3.el6_5.x86_64. >2014-09-09 17:36:09,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 21/113: 1:xorg-x11-font-utils-7.2-11.el6.x86_64 >2014-09-09 17:36:09,326 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 21/113: 1:xorg-x11-font-utils-7.2-11.el6.x86_64. >2014-09-09 17:36:09,726 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 22/113: ghostscript-fonts-5.50-23.1.el6.noarch >2014-09-09 17:36:09,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 22/113: ghostscript-fonts-5.50-23.1.el6.noarch. >2014-09-09 17:36:12,619 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 23/113: rpcbind-0.2.0-11.el6.x86_64 >2014-09-09 17:36:12,635 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 23/113: rpcbind-0.2.0-11.el6.x86_64. >2014-09-09 17:36:13,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 24/113: numad-0.5-10.20140620git.el6_5.x86_64 >2014-09-09 17:36:13,043 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 24/113: numad-0.5-10.20140620git.el6_5.x86_64. >2014-09-09 17:36:13,473 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 25/113: sanlock-python-2.8-1.el6.x86_64 >2014-09-09 17:36:13,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 25/113: sanlock-python-2.8-1.el6.x86_64. >2014-09-09 17:36:13,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 26/113: augeas-1.0.0-5.el6_5.1.x86_64 >2014-09-09 17:36:13,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 26/113: augeas-1.0.0-5.el6_5.1.x86_64. >2014-09-09 17:36:14,298 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 27/113: netcf-libs-0.1.9-4.el6_5.2.x86_64 >2014-09-09 17:36:14,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 27/113: netcf-libs-0.1.9-4.el6_5.2.x86_64. >2014-09-09 17:36:14,824 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 28/113: celt051-0.5.1.3-0.el6.x86_64 >2014-09-09 17:36:14,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 28/113: celt051-0.5.1.3-0.el6.x86_64. >2014-09-09 17:36:15,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 29/113: spice-server-0.12.4-6.el6_5.2.x86_64 >2014-09-09 17:36:15,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 29/113: spice-server-0.12.4-6.el6_5.2.x86_64. >2014-09-09 17:36:15,762 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 30/113: flac-1.2.1-6.1.el6.x86_64 >2014-09-09 17:36:15,775 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 30/113: flac-1.2.1-6.1.el6.x86_64. >2014-09-09 17:36:16,198 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 31/113: 1:libvorbis-1.2.3-4.el6_2.1.x86_64 >2014-09-09 17:36:16,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 31/113: 1:libvorbis-1.2.3-4.el6_2.1.x86_64. >2014-09-09 17:36:16,631 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 32/113: libsndfile-1.0.20-5.el6.x86_64 >2014-09-09 17:36:16,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 32/113: libsndfile-1.0.20-5.el6.x86_64. >2014-09-09 17:36:17,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 33/113: glusterfs-3.5.2-1.el6.x86_64 >2014-09-09 17:36:17,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 33/113: glusterfs-3.5.2-1.el6.x86_64. >2014-09-09 17:36:17,716 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 34/113: glusterfs-api-3.5.2-1.el6.x86_64 >2014-09-09 17:36:17,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 34/113: glusterfs-api-3.5.2-1.el6.x86_64. >2014-09-09 17:36:18,157 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 35/113: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:36:18,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 35/113: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:36:18,545 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 36/113: gnutls-utils-2.8.5-14.el6_5.x86_64 >2014-09-09 17:36:18,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 36/113: gnutls-utils-2.8.5-14.el6_5.x86_64. >2014-09-09 17:36:18,909 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 37/113: ioprocess-0.12.0-2.el6.x86_64 >2014-09-09 17:36:18,922 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 37/113: ioprocess-0.12.0-2.el6.x86_64. >2014-09-09 17:36:19,248 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 38/113: python-ioprocess-0.12.0-2.el6.noarch >2014-09-09 17:36:19,260 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 38/113: python-ioprocess-0.12.0-2.el6.noarch. >2014-09-09 17:36:19,574 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 39/113: vdsm-python-4.16.3-2.el6.x86_64 >2014-09-09 17:36:19,589 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 39/113: vdsm-python-4.16.3-2.el6.x86_64. >2014-09-09 17:36:19,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 40/113: vdsm-xmlrpc-4.16.3-2.el6.noarch >2014-09-09 17:36:20,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 40/113: vdsm-xmlrpc-4.16.3-2.el6.noarch. >2014-09-09 17:36:20,301 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 41/113: vdsm-cli-4.16.3-2.el6.noarch >2014-09-09 17:36:20,315 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 41/113: vdsm-cli-4.16.3-2.el6.noarch. >2014-09-09 17:36:20,643 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 42/113: audit-libs-python-2.2-4.el6_5.x86_64 >2014-09-09 17:36:20,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 42/113: audit-libs-python-2.2-4.el6_5.x86_64. >2014-09-09 17:36:20,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 43/113: lm_sensors-libs-3.1.1-17.el6.x86_64 >2014-09-09 17:36:21,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 43/113: lm_sensors-libs-3.1.1-17.el6.x86_64. >2014-09-09 17:36:21,419 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 44/113: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64 >2014-09-09 17:36:21,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 44/113: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64. >2014-09-09 17:36:22,050 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 45/113: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64 >2014-09-09 17:36:22,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 45/113: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64. >2014-09-09 17:36:22,476 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 46/113: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 >2014-09-09 17:36:22,488 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 46/113: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64. >2014-09-09 17:36:22,847 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 47/113: ipmitool-1.8.11-16.el6.x86_64 >2014-09-09 17:36:22,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 47/113: ipmitool-1.8.11-16.el6.x86_64. >2014-09-09 17:36:23,340 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 48/113: libgomp-4.4.7-4.el6.x86_64 >2014-09-09 17:36:23,355 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 48/113: libgomp-4.4.7-4.el6.x86_64. >2014-09-09 17:36:23,782 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 49/113: python-netaddr-0.7.5-4.el6.noarch >2014-09-09 17:36:23,795 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 49/113: python-netaddr-0.7.5-4.el6.noarch. >2014-09-09 17:36:24,281 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 50/113: dnsmasq-2.48-13.el6.x86_64 >2014-09-09 17:36:24,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 50/113: dnsmasq-2.48-13.el6.x86_64. >2014-09-09 17:36:24,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 51/113: python-ordereddict-1.1-3.el6ev.noarch >2014-09-09 17:36:24,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 51/113: python-ordereddict-1.1-3.el6ev.noarch. >2014-09-09 17:36:24,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 52/113: sg3_utils-1.28-5.el6.x86_64 >2014-09-09 17:36:24,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 52/113: sg3_utils-1.28-5.el6.x86_64. >2014-09-09 17:36:25,415 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 53/113: python-argparse-1.2.1-2.1.el6.noarch >2014-09-09 17:36:25,436 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 53/113: python-argparse-1.2.1-2.1.el6.noarch. >2014-09-09 17:36:25,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 54/113: cvs-1.11.23-16.el6.x86_64 >2014-09-09 17:36:25,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 54/113: cvs-1.11.23-16.el6.x86_64. >2014-09-09 17:36:26,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 55/113: gettext-0.17-16.el6.x86_64 >2014-09-09 17:36:26,246 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 55/113: gettext-0.17-16.el6.x86_64. >2014-09-09 17:36:27,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 56/113: perl-Net-Telnet-3.03-11.el6.noarch >2014-09-09 17:36:27,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 56/113: perl-Net-Telnet-3.03-11.el6.noarch. >2014-09-09 17:36:27,356 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 57/113: libXtst-1.2.1-2.el6.x86_64 >2014-09-09 17:36:27,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 57/113: libXtst-1.2.1-2.el6.x86_64. >2014-09-09 17:36:27,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 58/113: seabios-0.6.1.2-28.el6.x86_64 >2014-09-09 17:36:27,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 58/113: seabios-0.6.1.2-28.el6.x86_64. >2014-09-09 17:36:27,998 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 59/113: python-suds-0.4.1-3.el6.noarch >2014-09-09 17:36:28,012 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 59/113: python-suds-0.4.1-3.el6.noarch. >2014-09-09 17:36:28,462 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 60/113: 1:telnet-0.17-47.el6_3.1.x86_64 >2014-09-09 17:36:28,478 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 60/113: 1:telnet-0.17-47.el6_3.1.x86_64. >2014-09-09 17:36:28,831 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 61/113: urw-fonts-2.4-10.el6.noarch >2014-09-09 17:36:28,845 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 61/113: urw-fonts-2.4-10.el6.noarch. >2014-09-09 17:36:31,678 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 62/113: ghostscript-8.70-19.el6.x86_64 >2014-09-09 17:36:31,693 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 62/113: ghostscript-8.70-19.el6.x86_64. >2014-09-09 17:36:32,906 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 63/113: libevent-1.4.13-4.el6.x86_64 >2014-09-09 17:36:32,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 63/113: libevent-1.4.13-4.el6.x86_64. >2014-09-09 17:36:33,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 64/113: python-inotify-0.9.1-1.el6ev.noarch >2014-09-09 17:36:33,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 64/113: python-inotify-0.9.1-1.el6ev.noarch. >2014-09-09 17:36:33,706 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 65/113: ebtables-2.0.9-6.el6.x86_64 >2014-09-09 17:36:33,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 65/113: ebtables-2.0.9-6.el6.x86_64. >2014-09-09 17:36:34,165 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 66/113: pexpect-2.3-6.el6.noarch >2014-09-09 17:36:34,185 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 66/113: pexpect-2.3-6.el6.noarch. >2014-09-09 17:36:34,574 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 67/113: keyutils-1.4-4.el6.x86_64 >2014-09-09 17:36:34,584 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 67/113: keyutils-1.4-4.el6.x86_64. >2014-09-09 17:36:34,906 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 68/113: nfs-utils-lib-1.1.5-6.el6_5.x86_64 >2014-09-09 17:36:34,917 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 68/113: nfs-utils-lib-1.1.5-6.el6_5.x86_64. >2014-09-09 17:36:35,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 69/113: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64 >2014-09-09 17:36:35,869 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 69/113: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64. >2014-09-09 17:36:36,662 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 70/113: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64 >2014-09-09 17:36:36,678 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 70/113: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64. >2014-09-09 17:36:37,120 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 71/113: genisoimage-1.1.9-12.el6.x86_64 >2014-09-09 17:36:37,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 71/113: genisoimage-1.1.9-12.el6.x86_64. >2014-09-09 17:36:37,567 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 72/113: scrub-2.2-1.el6.x86_64 >2014-09-09 17:36:37,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 72/113: scrub-2.2-1.el6.x86_64. >2014-09-09 17:36:37,889 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 73/113: setools-libs-3.3.7-4.el6.x86_64 >2014-09-09 17:36:37,903 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 73/113: setools-libs-3.3.7-4.el6.x86_64. >2014-09-09 17:36:38,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 74/113: setools-libs-python-3.3.7-4.el6.x86_64 >2014-09-09 17:36:38,384 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 74/113: setools-libs-python-3.3.7-4.el6.x86_64. >2014-09-09 17:36:39,116 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 75/113: radvd-1.6-1.el6.x86_64 >2014-09-09 17:36:39,128 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 75/113: radvd-1.6-1.el6.x86_64. >2014-09-09 17:36:39,539 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 76/113: libasyncns-0.8-1.1.el6.x86_64 >2014-09-09 17:36:39,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 76/113: libasyncns-0.8-1.1.el6.x86_64. >2014-09-09 17:36:39,931 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 77/113: pulseaudio-libs-0.9.21-14.el6_3.x86_64 >2014-09-09 17:36:39,948 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 77/113: pulseaudio-libs-0.9.21-14.el6_3.x86_64. >2014-09-09 17:36:40,467 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 78/113: kpartx-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:36:40,481 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 78/113: kpartx-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:36:40,837 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 79/113: febootstrap-supermin-helper-3.21-4.el6.x86_64 >2014-09-09 17:36:40,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 79/113: febootstrap-supermin-helper-3.21-4.el6.x86_64. >2014-09-09 17:36:41,222 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 80/113: python-decorator-3.0.1-3.1.el6.noarch >2014-09-09 17:36:41,238 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 80/113: python-decorator-3.0.1-3.1.el6.noarch. >2014-09-09 17:36:41,701 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 81/113: pyparted-3.4-4.el6.x86_64 >2014-09-09 17:36:41,718 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 81/113: pyparted-3.4-4.el6.x86_64. >2014-09-09 17:36:42,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 82/113: libsemanage-python-2.0.43-4.2.el6.x86_64 >2014-09-09 17:36:42,123 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 82/113: libsemanage-python-2.0.43-4.2.el6.x86_64. >2014-09-09 17:36:42,460 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 83/113: policycoreutils-python-2.0.83-19.39.el6.x86_64 >2014-09-09 17:36:42,474 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 83/113: policycoreutils-python-2.0.83-19.39.el6.x86_64. >2014-09-09 17:37:10,753 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 84/113: fuse-2.8.3-4.el6.x86_64 >2014-09-09 17:37:10,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 84/113: fuse-2.8.3-4.el6.x86_64. >2014-09-09 17:37:11,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 85/113: vdsm-yajsonrpc-4.16.3-2.el6.noarch >2014-09-09 17:37:11,179 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 85/113: vdsm-yajsonrpc-4.16.3-2.el6.noarch. >2014-09-09 17:37:11,504 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 86/113: vdsm-jsonrpc-4.16.3-2.el6.noarch >2014-09-09 17:37:11,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 86/113: vdsm-jsonrpc-4.16.3-2.el6.noarch. >2014-09-09 17:37:12,130 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 87/113: python-pthreading-0.1.3-3.el6ev.noarch >2014-09-09 17:37:12,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 87/113: python-pthreading-0.1.3-3.el6ev.noarch. >2014-09-09 17:37:12,523 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 88/113: nc-1.84-22.el6.x86_64 >2014-09-09 17:37:12,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 88/113: nc-1.84-22.el6.x86_64. >2014-09-09 17:37:12,917 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 89/113: libvirt-client-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:37:12,932 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 89/113: libvirt-client-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:37:14,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 90/113: libvirt-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:37:14,440 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 90/113: libvirt-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:37:15,184 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 91/113: libvirt-python-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:37:15,197 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 91/113: libvirt-python-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:37:15,631 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 92/113: mom-0.4.1-2.el6.noarch >2014-09-09 17:37:15,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 92/113: mom-0.4.1-2.el6.noarch. >2014-09-09 17:37:16,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 93/113: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 >2014-09-09 17:37:16,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 93/113: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64. >2014-09-09 17:37:16,540 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 94/113: fence-agents-3.1.5-35.el6_5.4.x86_64 >2014-09-09 17:37:16,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 94/113: fence-agents-3.1.5-35.el6_5.4.x86_64. >2014-09-09 17:37:17,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 95/113: netpbm-10.47.05-11.el6.x86_64 >2014-09-09 17:37:17,432 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 95/113: netpbm-10.47.05-11.el6.x86_64. >2014-09-09 17:37:18,123 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 96/113: netpbm-progs-10.47.05-11.el6.x86_64 >2014-09-09 17:37:18,137 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 96/113: netpbm-progs-10.47.05-11.el6.x86_64. >2014-09-09 17:37:18,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 97/113: gpxe-roms-qemu-0.9.7-6.10.el6.noarch >2014-09-09 17:37:18,968 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 97/113: gpxe-roms-qemu-0.9.7-6.10.el6.noarch. >2014-09-09 17:37:19,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 98/113: sgabios-bin-0-0.3.20110621svn.el6.noarch >2014-09-09 17:37:19,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 98/113: sgabios-bin-0-0.3.20110621svn.el6.noarch. >2014-09-09 17:37:19,624 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 99/113: tree-1.5.3-2.el6.x86_64 >2014-09-09 17:37:19,639 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 99/113: tree-1.5.3-2.el6.x86_64. >2014-09-09 17:37:19,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 100/113: vgabios-0.6b-3.7.el6.noarch >2014-09-09 17:37:19,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 100/113: vgabios-0.6b-3.7.el6.noarch. >2014-09-09 17:37:20,319 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 101/113: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:37:20,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 101/113: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:37:21,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 102/113: 1:libguestfs-1.20.11-2.el6.x86_64 >2014-09-09 17:37:21,122 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 102/113: 1:libguestfs-1.20.11-2.el6.x86_64. >2014-09-09 17:37:21,754 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 103/113: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:37:21,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 103/113: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:37:22,271 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 104/113: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64 >2014-09-09 17:37:22,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 104/113: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64. >2014-09-09 17:37:22,746 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 105/113: libconfig-1.3.2-1.1.el6.x86_64 >2014-09-09 17:37:22,759 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 105/113: libconfig-1.3.2-1.1.el6.x86_64. >2014-09-09 17:37:23,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 106/113: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64 >2014-09-09 17:37:23,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 106/113: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64. >2014-09-09 17:37:24,220 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 107/113: vdsm-4.16.3-2.el6.x86_64 >2014-09-09 17:37:24,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 107/113: vdsm-4.16.3-2.el6.x86_64. >2014-09-09 17:37:48,215 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum update: 108/113: audit-2.2-4.el6_5.x86_64 >2014-09-09 17:37:48,230 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum update: 108/113: audit-2.2-4.el6_5.x86_64. >2014-09-09 17:37:48,784 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 109/113: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-09 17:37:48,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 109/113: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-09 17:37:49,165 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 110/113: audit >2014-09-09 17:37:49,175 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 110/113: audit. >2014-09-09 17:37:49,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 111/113: audit-libs >2014-09-09 17:37:49,801 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 111/113: audit-libs. >2014-09-09 17:37:50,106 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 112/113: kpartx >2014-09-09 17:37:50,119 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 112/113: kpartx. >2014-09-09 17:37:50,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum updated: 113/113: gnutls >2014-09-09 17:37:50,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum updated: 113/113: gnutls. >2014-09-09 17:38:02,755 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/113: libconfig.x86_64 0:1.3.2-1.1.el6 - u >2014-09-09 17:38:02,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/113: libconfig.x86_64 0:1.3.2-1.1.el6 - u. >2014-09-09 17:38:02,824 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 2/113: device-mapper-multipath-libs.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-09 17:38:02,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 2/113: device-mapper-multipath-libs.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-09 17:38:02,916 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 3/113: vgabios.noarch 0:0.6b-3.7.el6 - u >2014-09-09 17:38:02,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 3/113: vgabios.noarch 0:0.6b-3.7.el6 - u. >2014-09-09 17:38:03,009 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 4/113: tree.x86_64 0:1.5.3-2.el6 - u >2014-09-09 17:38:03,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 4/113: tree.x86_64 0:1.5.3-2.el6 - u. >2014-09-09 17:38:03,082 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 5/113: sgabios-bin.noarch 0:0-0.3.20110621svn.el6 - u >2014-09-09 17:38:03,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 5/113: sgabios-bin.noarch 0:0-0.3.20110621svn.el6 - u. >2014-09-09 17:38:03,194 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 6/113: qemu-img-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-09 17:38:03,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 6/113: qemu-img-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-09 17:38:03,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 7/113: celt051.x86_64 0:0.5.1.3-0.el6 - u >2014-09-09 17:38:03,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 7/113: celt051.x86_64 0:0.5.1.3-0.el6 - u. >2014-09-09 17:38:03,386 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 8/113: sanlock.x86_64 0:2.8-1.el6 - u >2014-09-09 17:38:03,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 8/113: sanlock.x86_64 0:2.8-1.el6 - u. >2014-09-09 17:38:03,479 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 9/113: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-09 17:38:03,494 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 9/113: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-09 17:38:03,571 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 10/113: hivex.x86_64 0:1.3.3-4.2.el6 - u >2014-09-09 17:38:03,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 10/113: hivex.x86_64 0:1.3.3-4.2.el6 - u. >2014-09-09 17:38:03,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 11/113: ghostscript-fonts.noarch 0:5.50-23.1.el6 - u >2014-09-09 17:38:03,679 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 11/113: ghostscript-fonts.noarch 0:5.50-23.1.el6 - u. >2014-09-09 17:38:03,715 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 12/113: gettext.x86_64 0:0.17-16.el6 - u >2014-09-09 17:38:03,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 12/113: gettext.x86_64 0:0.17-16.el6 - u. >2014-09-09 17:38:03,808 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 13/113: gpxe-roms-qemu.noarch 0:0.9.7-6.10.el6 - u >2014-09-09 17:38:03,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 13/113: gpxe-roms-qemu.noarch 0:0.9.7-6.10.el6 - u. >2014-09-09 17:38:03,873 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 14/113: netpbm.x86_64 0:10.47.05-11.el6 - u >2014-09-09 17:38:03,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 14/113: netpbm.x86_64 0:10.47.05-11.el6 - u. >2014-09-09 17:38:03,968 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 15/113: gnutls.x86_64 0:2.8.5-14.el6_5 - u >2014-09-09 17:38:03,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 15/113: gnutls.x86_64 0:2.8.5-14.el6_5 - u. >2014-09-09 17:38:04,060 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 16/113: nc.x86_64 0:1.84-22.el6 - u >2014-09-09 17:38:04,071 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 16/113: nc.x86_64 0:1.84-22.el6 - u. >2014-09-09 17:38:04,169 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 17/113: mom.noarch 0:0.4.1-2.el6 - u >2014-09-09 17:38:04,183 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 17/113: mom.noarch 0:0.4.1-2.el6 - u. >2014-09-09 17:38:04,237 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 18/113: pyparted.x86_64 0:3.4-4.el6 - u >2014-09-09 17:38:04,251 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 18/113: pyparted.x86_64 0:3.4-4.el6 - u. >2014-09-09 17:38:04,290 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 19/113: python-pthreading.noarch 0:0.1.3-3.el6ev - u >2014-09-09 17:38:04,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 19/113: python-pthreading.noarch 0:0.1.3-3.el6ev - u. >2014-09-09 17:38:04,380 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 20/113: vdsm-yajsonrpc.noarch 0:4.16.3-2.el6 - u >2014-09-09 17:38:04,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 20/113: vdsm-yajsonrpc.noarch 0:4.16.3-2.el6 - u. >2014-09-09 17:38:04,456 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 21/113: fuse.x86_64 0:2.8.3-4.el6 - u >2014-09-09 17:38:04,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 21/113: fuse.x86_64 0:2.8.3-4.el6 - u. >2014-09-09 17:38:04,558 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 22/113: gnutls-utils.x86_64 0:2.8.5-14.el6_5 - u >2014-09-09 17:38:04,571 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 22/113: gnutls-utils.x86_64 0:2.8.5-14.el6_5 - u. >2014-09-09 17:38:04,607 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 23/113: libsemanage-python.x86_64 0:2.0.43-4.2.el6 - u >2014-09-09 17:38:04,620 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 23/113: libsemanage-python.x86_64 0:2.0.43-4.2.el6 - u. >2014-09-09 17:38:04,701 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 24/113: libvirt.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-09 17:38:04,715 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 24/113: libvirt.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-09 17:38:04,802 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 25/113: python-decorator.noarch 0:3.0.1-3.1.el6 - u >2014-09-09 17:38:04,830 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 25/113: python-decorator.noarch 0:3.0.1-3.1.el6 - u. >2014-09-09 17:38:04,886 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 26/113: flac.x86_64 0:1.2.1-6.1.el6 - u >2014-09-09 17:38:04,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 26/113: flac.x86_64 0:1.2.1-6.1.el6 - u. >2014-09-09 17:38:04,987 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 27/113: febootstrap-supermin-helper.x86_64 0:3.21-4.el6 - u >2014-09-09 17:38:05,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 27/113: febootstrap-supermin-helper.x86_64 0:3.21-4.el6 - u. >2014-09-09 17:38:05,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 28/113: vdsm-python-zombiereaper.noarch 0:4.16.3-2.el6 - u >2014-09-09 17:38:05,062 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 28/113: vdsm-python-zombiereaper.noarch 0:4.16.3-2.el6 - u. >2014-09-09 17:38:05,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 29/113: kpartx.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-09 17:38:05,125 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 29/113: kpartx.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-09 17:38:05,180 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 30/113: libasyncns.x86_64 0:0.8-1.1.el6 - u >2014-09-09 17:38:05,194 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 30/113: libasyncns.x86_64 0:0.8-1.1.el6 - u. >2014-09-09 17:38:05,273 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 31/113: radvd.x86_64 0:1.6-1.el6 - u >2014-09-09 17:38:05,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 31/113: radvd.x86_64 0:1.6-1.el6 - u. >2014-09-09 17:38:05,365 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 32/113: usbredir.x86_64 0:0.5.1-1.el6 - u >2014-09-09 17:38:05,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 32/113: usbredir.x86_64 0:0.5.1-1.el6 - u. >2014-09-09 17:38:05,459 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 33/113: setools-libs.x86_64 0:3.3.7-4.el6 - u >2014-09-09 17:38:05,476 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 33/113: setools-libs.x86_64 0:3.3.7-4.el6 - u. >2014-09-09 17:38:05,515 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 34/113: vdsm-cli.noarch 0:4.16.3-2.el6 - u >2014-09-09 17:38:05,531 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 34/113: vdsm-cli.noarch 0:4.16.3-2.el6 - u. >2014-09-09 17:38:05,602 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 35/113: scrub.x86_64 0:2.2-1.el6 - u >2014-09-09 17:38:05,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 35/113: scrub.x86_64 0:2.2-1.el6 - u. >2014-09-09 17:38:05,678 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 36/113: genisoimage.x86_64 0:1.1.9-12.el6 - u >2014-09-09 17:38:05,688 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 36/113: genisoimage.x86_64 0:1.1.9-12.el6 - u. >2014-09-09 17:38:05,778 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 37/113: vdsm-python.x86_64 0:4.16.3-2.el6 - u >2014-09-09 17:38:05,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 37/113: vdsm-python.x86_64 0:4.16.3-2.el6 - u. >2014-09-09 17:38:05,854 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 38/113: btrfs-progs.x86_64 0:0.20-0.2.git91d9eec.el6 - u >2014-09-09 17:38:05,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 38/113: btrfs-progs.x86_64 0:0.20-0.2.git91d9eec.el6 - u. >2014-09-09 17:38:05,923 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 39/113: audit.x86_64 0:2.2-4.el6_5 - u >2014-09-09 17:38:05,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 39/113: audit.x86_64 0:2.2-4.el6_5 - u. >2014-09-09 17:38:06,030 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 40/113: libtirpc.x86_64 0:0.2.1-6.el6_5.2 - u >2014-09-09 17:38:06,041 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 40/113: libtirpc.x86_64 0:0.2.1-6.el6_5.2 - u. >2014-09-09 17:38:06,114 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 41/113: net-snmp-utils.x86_64 1:5.5-49.el6_5.2 - u >2014-09-09 17:38:06,128 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 41/113: net-snmp-utils.x86_64 1:5.5-49.el6_5.2 - u. >2014-09-09 17:38:06,207 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 42/113: rpcbind.x86_64 0:0.2.0-11.el6 - u >2014-09-09 17:38:06,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 42/113: rpcbind.x86_64 0:0.2.0-11.el6 - u. >2014-09-09 17:38:06,273 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 43/113: python-cpopen.x86_64 0:1.3-4.el6_5 - u >2014-09-09 17:38:06,288 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 43/113: python-cpopen.x86_64 0:1.3-4.el6_5 - u. >2014-09-09 17:38:06,376 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 44/113: qemu-kvm-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-09 17:38:06,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 44/113: qemu-kvm-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-09 17:38:06,448 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 45/113: numad.x86_64 0:0.5-10.20140620git.el6_5 - u >2014-09-09 17:38:06,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 45/113: numad.x86_64 0:0.5-10.20140620git.el6_5 - u. >2014-09-09 17:38:06,511 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 46/113: keyutils.x86_64 0:1.4-4.el6 - u >2014-09-09 17:38:06,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 46/113: keyutils.x86_64 0:1.4-4.el6 - u. >2014-09-09 17:38:06,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 47/113: augeas.x86_64 0:1.0.0-5.el6_5.1 - u >2014-09-09 17:38:06,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 47/113: augeas.x86_64 0:1.0.0-5.el6_5.1 - u. >2014-09-09 17:38:06,688 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 48/113: pexpect.noarch 0:2.3-6.el6 - u >2014-09-09 17:38:06,699 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 48/113: pexpect.noarch 0:2.3-6.el6 - u. >2014-09-09 17:38:06,780 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 49/113: ebtables.x86_64 0:2.0.9-6.el6 - u >2014-09-09 17:38:06,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 49/113: ebtables.x86_64 0:2.0.9-6.el6 - u. >2014-09-09 17:38:06,840 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 50/113: ghostscript.x86_64 0:8.70-19.el6 - u >2014-09-09 17:38:06,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 50/113: ghostscript.x86_64 0:8.70-19.el6 - u. >2014-09-09 17:38:06,915 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 51/113: libguestfs-tools-c.x86_64 1:1.20.11-2.el6 - u >2014-09-09 17:38:06,928 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 51/113: libguestfs-tools-c.x86_64 1:1.20.11-2.el6 - u. >2014-09-09 17:38:07,007 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 52/113: nfs-utils-lib.x86_64 0:1.1.5-6.el6_5 - u >2014-09-09 17:38:07,024 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 52/113: nfs-utils-lib.x86_64 0:1.1.5-6.el6_5 - u. >2014-09-09 17:38:07,082 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 53/113: libXfont.x86_64 0:1.4.5-3.el6_5 - u >2014-09-09 17:38:07,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 53/113: libXfont.x86_64 0:1.4.5-3.el6_5 - u. >2014-09-09 17:38:07,167 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 54/113: setools-libs-python.x86_64 0:3.3.7-4.el6 - u >2014-09-09 17:38:07,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 54/113: setools-libs-python.x86_64 0:3.3.7-4.el6 - u. >2014-09-09 17:38:07,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 55/113: vdsm-jsonrpc.noarch 0:4.16.3-2.el6 - u >2014-09-09 17:38:07,250 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 55/113: vdsm-jsonrpc.noarch 0:4.16.3-2.el6 - u. >2014-09-09 17:38:07,336 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 56/113: fence-agents.x86_64 0:3.1.5-35.el6_5.4 - u >2014-09-09 17:38:07,351 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 56/113: fence-agents.x86_64 0:3.1.5-35.el6_5.4 - u. >2014-09-09 17:38:07,421 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 57/113: python-inotify.noarch 0:0.9.1-1.el6ev - u >2014-09-09 17:38:07,436 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 57/113: python-inotify.noarch 0:0.9.1-1.el6ev - u. >2014-09-09 17:38:07,488 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 58/113: libevent.x86_64 0:1.4.13-4.el6 - u >2014-09-09 17:38:07,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 58/113: libevent.x86_64 0:1.4.13-4.el6 - u. >2014-09-09 17:38:07,572 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 59/113: urw-fonts.noarch 0:2.4-10.el6 - u >2014-09-09 17:38:07,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 59/113: urw-fonts.noarch 0:2.4-10.el6 - u. >2014-09-09 17:38:07,632 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 60/113: telnet.x86_64 1:0.17-47.el6_3.1 - u >2014-09-09 17:38:07,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 60/113: telnet.x86_64 1:0.17-47.el6_3.1 - u. >2014-09-09 17:38:07,732 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 61/113: audit-libs-python.x86_64 0:2.2-4.el6_5 - u >2014-09-09 17:38:07,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 61/113: audit-libs-python.x86_64 0:2.2-4.el6_5 - u. >2014-09-09 17:38:07,799 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 62/113: netcf-libs.x86_64 0:0.1.9-4.el6_5.2 - u >2014-09-09 17:38:07,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 62/113: netcf-libs.x86_64 0:0.1.9-4.el6_5.2 - u. >2014-09-09 17:38:07,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 63/113: python-suds.noarch 0:0.4.1-3.el6 - u >2014-09-09 17:38:07,880 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 63/113: python-suds.noarch 0:0.4.1-3.el6 - u. >2014-09-09 17:38:07,959 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 64/113: libcgroup.x86_64 0:0.40.rc1-5.el6_5.1 - u >2014-09-09 17:38:07,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 64/113: libcgroup.x86_64 0:0.40.rc1-5.el6_5.1 - u. >2014-09-09 17:38:08,043 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 65/113: seabios.x86_64 0:0.6.1.2-28.el6 - u >2014-09-09 17:38:08,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 65/113: seabios.x86_64 0:0.6.1.2-28.el6 - u. >2014-09-09 17:38:08,127 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 66/113: libogg.x86_64 2:1.1.4-2.1.el6 - u >2014-09-09 17:38:08,140 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 66/113: libogg.x86_64 2:1.1.4-2.1.el6 - u. >2014-09-09 17:38:08,182 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 67/113: libXtst.x86_64 0:1.2.1-2.el6 - u >2014-09-09 17:38:08,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 67/113: libXtst.x86_64 0:1.2.1-2.el6 - u. >2014-09-09 17:38:08,240 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 68/113: fuse-libs.x86_64 0:2.8.3-4.el6 - u >2014-09-09 17:38:08,255 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 68/113: fuse-libs.x86_64 0:2.8.3-4.el6 - u. >2014-09-09 17:38:08,322 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 69/113: libsndfile.x86_64 0:1.0.20-5.el6 - u >2014-09-09 17:38:08,335 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 69/113: libsndfile.x86_64 0:1.0.20-5.el6 - u. >2014-09-09 17:38:08,390 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 70/113: perl-Net-Telnet.noarch 0:3.03-11.el6 - u >2014-09-09 17:38:08,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 70/113: perl-Net-Telnet.noarch 0:3.03-11.el6 - u. >2014-09-09 17:38:08,473 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 71/113: net-snmp-libs.x86_64 1:5.5-49.el6_5.2 - u >2014-09-09 17:38:08,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 71/113: net-snmp-libs.x86_64 1:5.5-49.el6_5.2 - u. >2014-09-09 17:38:08,566 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 72/113: policycoreutils-python.x86_64 0:2.0.83-19.39.el6 - u >2014-09-09 17:38:08,579 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 72/113: policycoreutils-python.x86_64 0:2.0.83-19.39.el6 - u. >2014-09-09 17:38:08,658 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 73/113: yajl.x86_64 0:1.0.7-3.el6 - u >2014-09-09 17:38:08,672 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 73/113: yajl.x86_64 0:1.0.7-3.el6 - u. >2014-09-09 17:38:08,759 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 74/113: cvs.x86_64 0:1.11.23-16.el6 - u >2014-09-09 17:38:08,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 74/113: cvs.x86_64 0:1.11.23-16.el6 - u. >2014-09-09 17:38:08,815 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 75/113: nfs-utils.x86_64 1:1.2.3-39.el6_5.3 - u >2014-09-09 17:38:08,829 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 75/113: nfs-utils.x86_64 1:1.2.3-39.el6_5.3 - u. >2014-09-09 17:38:08,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 76/113: audit-libs.x86_64 0:2.2-4.el6_5 - u >2014-09-09 17:38:08,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 76/113: audit-libs.x86_64 0:2.2-4.el6_5 - u. >2014-09-09 17:38:08,965 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 77/113: sanlock-lib.x86_64 0:2.8-1.el6 - u >2014-09-09 17:38:08,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 77/113: sanlock-lib.x86_64 0:2.8-1.el6 - u. >2014-09-09 17:38:09,054 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 78/113: python-ioprocess.noarch 0:0.12.0-2.el6 - u >2014-09-09 17:38:09,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 78/113: python-ioprocess.noarch 0:0.12.0-2.el6 - u. >2014-09-09 17:38:09,107 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 79/113: netpbm-progs.x86_64 0:10.47.05-11.el6 - u >2014-09-09 17:38:09,119 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 79/113: netpbm-progs.x86_64 0:10.47.05-11.el6 - u. >2014-09-09 17:38:09,157 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 80/113: python-argparse.noarch 0:1.2.1-2.1.el6 - u >2014-09-09 17:38:09,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 80/113: python-argparse.noarch 0:1.2.1-2.1.el6 - u. >2014-09-09 17:38:09,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 81/113: libvorbis.x86_64 1:1.2.3-4.el6_2.1 - u >2014-09-09 17:38:09,246 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 81/113: libvorbis.x86_64 1:1.2.3-4.el6_2.1 - u. >2014-09-09 17:38:09,290 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 82/113: sg3_utils.x86_64 0:1.28-5.el6 - u >2014-09-09 17:38:09,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 82/113: sg3_utils.x86_64 0:1.28-5.el6 - u. >2014-09-09 17:38:09,367 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 83/113: spice-server.x86_64 0:0.12.4-6.el6_5.2 - u >2014-09-09 17:38:09,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 83/113: spice-server.x86_64 0:0.12.4-6.el6_5.2 - u. >2014-09-09 17:38:09,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 84/113: glusterfs.x86_64 0:3.5.2-1.el6 - u >2014-09-09 17:38:09,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 84/113: glusterfs.x86_64 0:3.5.2-1.el6 - u. >2014-09-09 17:38:09,518 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 85/113: perl-hivex.x86_64 0:1.3.3-4.2.el6 - u >2014-09-09 17:38:09,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 85/113: perl-hivex.x86_64 0:1.3.3-4.2.el6 - u. >2014-09-09 17:38:09,619 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 86/113: pulseaudio-libs.x86_64 0:0.9.21-14.el6_3 - u >2014-09-09 17:38:09,630 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 86/113: pulseaudio-libs.x86_64 0:0.9.21-14.el6_3 - u. >2014-09-09 17:38:09,674 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 87/113: python-ordereddict.noarch 0:1.1-3.el6ev - u >2014-09-09 17:38:09,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 87/113: python-ordereddict.noarch 0:1.1-3.el6ev - u. >2014-09-09 17:38:09,754 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 88/113: libfontenc.x86_64 0:1.0.5-2.el6 - u >2014-09-09 17:38:09,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 88/113: libfontenc.x86_64 0:1.0.5-2.el6 - u. >2014-09-09 17:38:09,821 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 89/113: xorg-x11-font-utils.x86_64 1:7.2-11.el6 - u >2014-09-09 17:38:09,835 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 89/113: xorg-x11-font-utils.x86_64 1:7.2-11.el6 - u. >2014-09-09 17:38:09,873 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 90/113: ioprocess.x86_64 0:0.12.0-2.el6 - u >2014-09-09 17:38:09,888 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 90/113: ioprocess.x86_64 0:0.12.0-2.el6 - u. >2014-09-09 17:38:09,958 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 91/113: lzop.x86_64 0:1.02-0.9.rc1.el6 - u >2014-09-09 17:38:09,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 91/113: lzop.x86_64 0:1.02-0.9.rc1.el6 - u. >2014-09-09 17:38:10,016 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 92/113: libvirt-client.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-09 17:38:10,031 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 92/113: libvirt-client.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-09 17:38:10,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 93/113: vdsm-xmlrpc.noarch 0:4.16.3-2.el6 - u >2014-09-09 17:38:10,114 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 93/113: vdsm-xmlrpc.noarch 0:4.16.3-2.el6 - u. >2014-09-09 17:38:10,157 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 94/113: sanlock-python.x86_64 0:2.8-1.el6 - u >2014-09-09 17:38:10,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 94/113: sanlock-python.x86_64 0:2.8-1.el6 - u. >2014-09-09 17:38:10,224 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 95/113: glusterfs-api.x86_64 0:3.5.2-1.el6 - u >2014-09-09 17:38:10,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 95/113: glusterfs-api.x86_64 0:3.5.2-1.el6 - u. >2014-09-09 17:38:10,313 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 96/113: dnsmasq.x86_64 0:2.48-13.el6 - u >2014-09-09 17:38:10,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 96/113: dnsmasq.x86_64 0:2.48-13.el6 - u. >2014-09-09 17:38:10,387 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 97/113: python-netaddr.noarch 0:0.7.5-4.el6 - u >2014-09-09 17:38:10,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 97/113: python-netaddr.noarch 0:0.7.5-4.el6 - u. >2014-09-09 17:38:10,463 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 98/113: libgomp.x86_64 0:4.4.7-4.el6 - u >2014-09-09 17:38:10,476 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 98/113: libgomp.x86_64 0:4.4.7-4.el6 - u. >2014-09-09 17:38:10,515 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 99/113: device-mapper-multipath.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-09 17:38:10,529 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 99/113: device-mapper-multipath.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-09 17:38:10,606 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 100/113: ipmitool.x86_64 0:1.8.11-16.el6 - u >2014-09-09 17:38:10,620 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 100/113: ipmitool.x86_64 0:1.8.11-16.el6 - u. >2014-09-09 17:38:10,674 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 101/113: cyrus-sasl-md5.x86_64 0:2.1.23-13.el6_3.1 - u >2014-09-09 17:38:10,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 101/113: cyrus-sasl-md5.x86_64 0:2.1.23-13.el6_3.1 - u. >2014-09-09 17:38:10,732 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 102/113: augeas-libs.x86_64 0:1.0.0-5.el6_5.1 - u >2014-09-09 17:38:10,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 102/113: augeas-libs.x86_64 0:1.0.0-5.el6_5.1 - u. >2014-09-09 17:38:10,818 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 103/113: lm_sensors-libs.x86_64 0:3.1.1-17.el6 - u >2014-09-09 17:38:10,833 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 103/113: lm_sensors-libs.x86_64 0:3.1.1-17.el6 - u. >2014-09-09 17:38:10,893 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 104/113: libguestfs.x86_64 1:1.20.11-2.el6 - u >2014-09-09 17:38:10,907 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 104/113: libguestfs.x86_64 1:1.20.11-2.el6 - u. >2014-09-09 17:38:10,985 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 105/113: glusterfs-libs.x86_64 0:3.5.2-1.el6 - u >2014-09-09 17:38:11,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 105/113: glusterfs-libs.x86_64 0:3.5.2-1.el6 - u. >2014-09-09 17:38:11,070 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 106/113: libgssglue.x86_64 0:0.1-11.el6 - u >2014-09-09 17:38:11,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 106/113: libgssglue.x86_64 0:0.1-11.el6 - u. >2014-09-09 17:38:11,132 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 107/113: libvirt-lock-sanlock.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-09 17:38:11,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 107/113: libvirt-lock-sanlock.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-09 17:38:11,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 108/113: vdsm.x86_64 0:4.16.3-2.el6 - u >2014-09-09 17:38:11,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 108/113: vdsm.x86_64 0:4.16.3-2.el6 - u. >2014-09-09 17:38:11,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 109/113: libvirt-python.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-09 17:38:11,271 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 109/113: libvirt-python.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-09 17:38:11,356 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 110/113: kpartx.x86_64 0:0.4.9-72.el6 - ud >2014-09-09 17:38:11,372 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 110/113: kpartx.x86_64 0:0.4.9-72.el6 - ud. >2014-09-09 17:38:11,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 111/113: audit.x86_64 0:2.2-2.el6 - ud >2014-09-09 17:38:11,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 111/113: audit.x86_64 0:2.2-2.el6 - ud. >2014-09-09 17:38:11,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 112/113: audit-libs.x86_64 0:2.2-2.el6 - ud >2014-09-09 17:38:11,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 112/113: audit-libs.x86_64 0:2.2-2.el6 - ud. >2014-09-09 17:38:11,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 113/113: gnutls.x86_64 0:2.8.5-10.el6_4.2 - ud >2014-09-09 17:38:11,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 113/113: gnutls.x86_64 0:2.8.5-10.el6_4.2 - ud. >2014-09-09 17:38:11,515 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-09 17:38:11,526 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-09 17:38:11,622 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-09 17:38:11,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-09 17:38:13,488 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-09 17:38:13,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-09 17:38:14,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-09 17:38:14,807 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-09 17:38:14,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-09 17:38:14,968 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-09 17:38:15,908 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-09 17:38:15,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-09 17:38:15,950 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-09 17:38:15,962 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-09 17:38:19,192 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-09 17:38:19,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-09 17:38:19,229 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909173819-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-6093dd15.log' >2014-09-09 17:38:19,246 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140909173819-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-6093dd15.log'. >2014-09-09 17:38:19,670 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-09 17:38:19,684 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6093dd15, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-09 17:38:19,841 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:38:20,070 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-36) [6093dd15] Engine managed to communicate with VDSM agent on host rack >2014-09-09 17:38:20,780 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-09 17:38:20,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] START, SetupNetworksVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=c97613e8-b860-495b-9207-7715533de9eb, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=ca117a0b-8f32-4c38-8b4c-aa15bd19f786, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 4c9c1b2b >2014-09-09 17:38:20,807 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] FINISH, SetupNetworksVDSCommand, log id: 4c9c1b2b >2014-09-09 17:41:21,122 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:41:21,123 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Command SetupNetworksVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=c97613e8-b860-495b-9207-7715533de9eb, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=ca117a0b-8f32-4c38-8b4c-aa15bd19f786, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:41:21,124 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' (Failed with error VDS_NETWORK_ERROR and code 5022) >2014-09-09 17:41:21,140 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Failed to configure management network: Failed to configure management network on host rack due to setup networks failure. >2014-09-09 17:41:21,144 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Host installation failed for host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, rack.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:206) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor78.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view8.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: Failed to configure management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:82) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:198) [bll.jar:] > ... 69 more > >2014-09-09 17:41:21,150 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 32132254 >2014-09-09 17:41:21,181 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] FINISH, SetVdsStatusVDSCommand, log id: 32132254 >2014-09-09 17:41:21,200 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Correlation ID: 6093dd15, Job ID: 28ad829d-a61d-4309-a303-4bf2b7ff5d54, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. Failed to configure management network on the host. >2014-09-09 17:41:21,214 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-36) [7bcb399f] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-09 17:45:00,042 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-11) Autorecovering 1 hosts >2014-09-09 17:45:00,042 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-11) Autorecovering hosts id: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, name : rack >2014-09-09 17:45:00,049 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-09 17:45:00,049 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:45:00,049 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-09 17:45:00,049 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-09 17:45:00,055 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5101d9e2 >2014-09-09 17:45:00,085 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] FINISH, SetVdsStatusVDSCommand, log id: 5101d9e2 >2014-09-09 17:45:00,122 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-09 17:45:00,135 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Correlation ID: 7b5e1817, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-09 17:45:00,135 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-11) [7b5e1817] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-09 17:45:01,973 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:45:01,975 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:295) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:203) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:144) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:45:15,474 WARN [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (ajp-/127.0.0.1:8702-4) [2872f9da] CanDoAction of action SetupNetworks failed. Reasons:VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,VAR__HOST_STATUS__UP_MAINTENANCE_OR_NON_OPERATIONAL,ACTION_TYPE_FAILED_VDS_STATUS_ILLEGAL >2014-09-09 17:47:23,254 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-1) [241f25cb] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 17:47:23,254 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-1) [241f25cb] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_VDS_NOT_MATCH_VALID_STATUS >2014-09-09 17:47:23,254 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-1) [241f25cb] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-09 17:47:52,266 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-47) Principal admin::internal synchronized >2014-09-09 17:47:52,600 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-09 17:48:04,263 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@7800906b >2014-09-09 17:48:04,263 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] HostName = rack >2014-09-09 17:48:04,263 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-30) [5b453678] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:48:04,263 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) [5b453678] Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:48:04,289 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [5b453678] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:48:04,311 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) [5b453678] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-09 17:48:04,318 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-30) [5b453678] Failed to refresh VDS , vds = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:48:07,356 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:48:07,357 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:265) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:474) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-09 17:51:09,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@194eb0ef >2014-09-09 17:51:09,827 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) HostName = rack >2014-09-09 17:51:09,827 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-73) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:51:09,828 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:51:09,874 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:51:09,874 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-09 17:51:09,880 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-73) Failed to refresh VDS , vds = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack, VDS Network Error, continuing. >VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:51:12,951 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-09 17:54:15,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@273ebe7a >2014-09-09 17:54:15,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) HostName = rack >2014-09-09 17:54:15,401 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-45) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:54:15,404 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:54:15,425 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-09 17:54:15,479 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Server failed to respond, vds_id = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-09 17:54:15,497 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-09 17:54:15,502 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-43) ResourceManager::vdsNotResponding entered for Host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:54:15,552 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-45) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-09 17:54:15,603 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-43) [138b849d] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-09 17:54:15,611 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-43) [138b849d] Opening SSH Soft Fencing session on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:54:15,767 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-43) [138b849d] Executing SSH Soft Fencing command on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-09 17:54:37,726 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-45) [2ce2a17e] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-09 17:54:37,857 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-45) [2ce2a17e] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-09 17:54:37,863 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-45) [2ce2a17e] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3a9093a7 >2014-09-09 18:47:52,282 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-80) Principal admin::internal synchronized >2014-09-09 18:47:52,608 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) [5b453678] Attempting to update VMs/Templates Ovf. >2014-09-09 19:47:52,296 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) [18a5c631] Principal admin::internal synchronized >2014-09-09 19:47:52,612 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-09-09 20:47:52,313 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-09 20:47:52,616 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) [8ff0b8] Attempting to update VMs/Templates Ovf. >2014-09-09 21:47:52,326 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-09-09 21:47:52,620 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) Attempting to update VMs/Templates Ovf. >2014-09-09 22:47:52,338 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-99) Principal admin::internal synchronized >2014-09-09 22:47:52,626 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-09-09 23:47:52,352 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-92) Principal admin::internal synchronized >2014-09-09 23:47:52,630 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-8) [13036fef] Attempting to update VMs/Templates Ovf. >2014-09-10 00:47:52,365 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-16) Principal admin::internal synchronized >2014-09-10 00:47:52,634 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-10 01:47:52,376 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-10 01:47:52,637 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) Attempting to update VMs/Templates Ovf. >2014-09-10 02:47:52,387 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Principal admin::internal synchronized >2014-09-10 02:47:52,640 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-09-10 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Start deleteAgedOutAuditLogs >2014-09-10 03:35:35,010 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-55) Start deleteAgedOutCommandEntities >2014-09-10 03:35:35,034 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Finished deleteAgedOutAuditLogs >2014-09-10 03:35:35,038 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-55) Finished deleteAgedOutCommandEntities >2014-09-10 03:47:52,399 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) Principal admin::internal synchronized >2014-09-10 03:47:52,644 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-09-10 04:47:52,411 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) Principal admin::internal synchronized >2014-09-10 04:47:52,647 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VMs/Templates Ovf. >2014-09-10 05:47:52,428 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-10 05:47:52,650 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) [5c5dd985] Attempting to update VMs/Templates Ovf. >2014-09-10 06:47:52,444 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-67) Principal admin::internal synchronized >2014-09-10 06:47:52,653 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) [24b7b64b] Attempting to update VMs/Templates Ovf. >2014-09-10 07:47:52,464 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-09-10 07:47:52,657 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-09-10 08:47:52,484 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-14) Principal admin::internal synchronized >2014-09-10 08:47:52,659 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2014-09-10 09:47:52,506 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-10 09:47:52,661 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-10 10:47:52,522 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) [b670564] Principal admin::internal synchronized >2014-09-10 10:47:52,664 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-69) [2f5064a8] Attempting to update VMs/Templates Ovf. >2014-09-10 10:57:22,294 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [6edc5c44] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,311 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [29632489] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,312 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [551c7763] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,321 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [5967886e] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,324 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [3fe2303c] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,376 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [551c7763] Correlation ID: 551c7763, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,384 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [1742c1b1] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,387 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [6edc5c44] Correlation ID: 6edc5c44, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [5967886e] Correlation ID: 5967886e, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [3fe2303c] Correlation ID: 3fe2303c, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,400 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [29632489] Correlation ID: 29632489, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1742c1b1] Correlation ID: 1742c1b1, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,420 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [6e85be44] Running command: LogoutUserCommand internal: false. >2014-09-10 10:57:22,434 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [6e85be44] Correlation ID: 6e85be44, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 10:57:22,499 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-10 10:57:22,499 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-10 10:58:34,566 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-10 10:58:34,567 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-10 11:01:52,170 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginAdminUserCommand internal: false. >2014-09-10 11:01:52,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 11:01:52,296 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-10 11:02:29,224 WARN [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (ajp-/127.0.0.1:8702-3) [7d081f43] CanDoAction of action SetupNetworks failed. Reasons:VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,VAR__HOST_STATUS__UP_MAINTENANCE_OR_NON_OPERATIONAL,ACTION_TYPE_FAILED_VDS_STATUS_ILLEGAL >2014-09-10 11:02:39,265 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-48) [6c118c1b] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:02:39,482 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-48) [6c118c1b] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:02:39,596 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-48) [6c118c1b] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4cb58e28 >2014-09-10 11:02:48,482 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-49) [45edf56d] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:02:48,687 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-49) [45edf56d] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:02:48,757 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [45edf56d] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 11707f33 >2014-09-10 11:03:00,603 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-50) [2233ebd5] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:03:00,743 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-50) [2233ebd5] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:03:00,755 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-50) [2233ebd5] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 387e1109 >2014-09-10 11:03:12,810 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-1) [71ce14f4] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:03:12,856 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-1) [71ce14f4] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:03:12,890 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-1) [71ce14f4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: fa65bc0 >2014-09-10 11:03:46,629 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-2) [58600e60] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:03:46,675 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-2) [58600e60] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:03:46,680 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-2) [58600e60] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 75a14fb >2014-09-10 11:04:08,533 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-2) [3d5e1254] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 11:04:08,543 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-6-thread-3) [3d5e1254] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:04:08,555 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (org.ovirt.thread.pool-6-thread-3) [39e2f8ca] Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:04:08,564 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-6-thread-3) [39e2f8ca] START, UpdateVdsVMsClearedVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: 30e21ebf >2014-09-10 11:04:17,298 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-4) [77f83a9a] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 11:04:17,349 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-4) [77f83a9a] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 11:04:17,354 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [77f83a9a] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7725d9ff >2014-09-10 11:04:43,769 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-5) [44f99934] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 11:04:43,770 WARN [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-5) [44f99934] CanDoAction of action FenceVdsManualy failed. Reasons:VAR__ACTION__MANUAL_FENCE,VAR__TYPE__HOST,POWER_MANAGEMENT_ACTION_ON_ENTITY_ALREADY_IN_PROGRESS >2014-09-10 11:14:08,552 ERROR [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (Transaction Reaper Worker 0) Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand. >2014-09-10 11:47:52,538 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) [18a5c631] Principal admin::internal synchronized >2014-09-10 11:47:52,668 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-99) Attempting to update VMs/Templates Ovf. >2014-09-10 12:21:59,203 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-10 12:21:59,203 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-10 12:26:22,938 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-6) [c82eb57] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:26:23,039 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-6) [c82eb57] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:26:23,046 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-6) [c82eb57] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 52ce6a41 >2014-09-10 12:26:41,951 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-58) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:82) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:26:41,997 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-68) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:26:46,958 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-52) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:82) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:26:47,003 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-76) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:26:58,780 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-10 12:26:58,781 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-10 12:26:58,783 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-10 12:26:58,785 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-10 12:26:58,786 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-10 12:26:58,795 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-10 12:26:58,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-10 12:26:58,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-10 12:26:58,796 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-10 12:26:58,797 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-10 12:26:58,797 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-10 12:26:58,797 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-10 12:26:58,798 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-10 12:26:58,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-10 12:26:58,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-10 12:26:58,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-10 12:26:58,799 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-10 12:26:58,803 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-10 12:26:58,803 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-10 12:26:58,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-10 12:26:58,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DB_USER" is "engine". >2014-09-10 12:26:58,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-10 12:26:58,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-10 12:26:58,804 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-10 12:26:58,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-10 12:26:58,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-10 12:26:58,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-10 12:26:58,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-10 12:26:58,805 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-10 12:26:58,806 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-10 12:26:58,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-10 12:26:58,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-10 12:26:58,807 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-10 12:26:58,815 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-10 12:26:58,816 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-10 12:26:58,816 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-10 12:26:58,816 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-10 12:26:58,816 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-10 12:26:58,816 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-10 12:26:58,824 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-10 12:26:58,825 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-10 12:26:58,826 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USER" is "ovirt". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-10 12:26:58,827 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-10 12:26:58,828 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 39) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-10 12:27:09,063 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Start initializing Backend >2014-09-10 12:27:09,691 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ajp-/127.0.0.1:8702-1) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-10 12:27:09,864 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Running ovirt-engine 3.5.0-0.10.master.el6ev >2014-09-10 12:27:09,865 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Start initializing dictionaries >2014-09-10 12:27:09,879 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ajp-/127.0.0.1:8702-1) Finished initializing dictionaries >2014-09-10 12:27:09,880 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing AuditLogCleanupManager >2014-09-10 12:27:09,880 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-10 12:27:09,918 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing AuditLogCleanupManager >2014-09-10 12:27:09,918 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing CommandEntityCleanupManager >2014-09-10 12:27:09,919 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-10 12:27:09,921 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing CommandEntityCleanupManager >2014-09-10 12:27:09,925 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Start initializing TagsDirector >2014-09-10 12:27:09,963 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Tag root added to tree >2014-09-10 12:27:09,985 INFO [org.ovirt.engine.core.bll.TagsDirector] (ajp-/127.0.0.1:8702-1) Finished initializing TagsDirector >2014-09-10 12:27:09,986 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Start initializing IsoDomainListSyncronizer >2014-09-10 12:27:09,998 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ajp-/127.0.0.1:8702-1) Finished initializing IsoDomainListSyncronizer >2014-09-10 12:27:10,074 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Completed initializing handlers >2014-09-10 12:27:10,124 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl >2014-09-10 12:27:10,154 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-10 12:27:10,156 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl >2014-09-10 12:27:10,156 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Start initializing ErrorTranslatorImpl >2014-09-10 12:27:10,158 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ajp-/127.0.0.1:8702-1) Finished initializing ErrorTranslatorImpl >2014-09-10 12:27:10,158 INFO [org.ovirt.engine.core.bll.Backend] (ajp-/127.0.0.1:8702-1) Mark incomplete jobs as UNKNOWN >2014-09-10 12:27:10,231 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Start initializing JobRepositoryCleanupManager >2014-09-10 12:27:10,232 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ajp-/127.0.0.1:8702-1) Finished initializing JobRepositoryCleanupManager >2014-09-10 12:27:10,232 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Start initializing AutoRecoveryManager >2014-09-10 12:27:10,234 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ajp-/127.0.0.1:8702-1) Finished initializing AutoRecoveryManager >2014-09-10 12:27:10,237 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Start initializing ExecutionMessageDirector >2014-09-10 12:27:10,248 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ajp-/127.0.0.1:8702-1) Finished initializing ExecutionMessageDirector >2014-09-10 12:27:10,463 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'builtin-authn-internal' >2014-09-10 12:27:10,470 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' loaded >2014-09-10 12:27:10,473 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Loading extension 'internal' >2014-09-10 12:27:10,476 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' loaded >2014-09-10 12:27:10,477 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'builtin-authn-internal' >2014-09-10 12:27:10,477 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'builtin-authn-internal' initialized >2014-09-10 12:27:10,480 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Initializing extension 'internal' >2014-09-10 12:27:10,480 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Extension 'internal' initialized >2014-09-10 12:27:10,480 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Start of enabled extensions list >2014-09-10 12:27:10,481 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-10 12:27:10,481 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-10 12:27:10,481 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ajp-/127.0.0.1:8702-1) End of enabled extensions list >2014-09-10 12:27:10,843 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-10 12:27:10,878 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 49) Start initializing DbUserCacheManager >2014-09-10 12:27:10,881 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-10 12:27:10,890 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 49) Finished initializing DbUserCacheManager >2014-09-10 12:27:10,950 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 49) Initialization of AsyncTaskManager completed successfully. >2014-09-10 12:27:10,963 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) Start initializing ResourceManager >2014-09-10 12:27:11,143 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-10 12:27:11,311 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Entered VdsManager constructor >2014-09-10 12:27:11,361 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-10 12:27:11,361 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-10 12:27:11,455 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 49) Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-10 12:27:11,804 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) VDS 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 was added to the Resource Manager >2014-09-10 12:27:11,861 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 49) Finished initializing ResourceManager >2014-09-10 12:27:11,869 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 49) Initialization of OvfDataUpdater completed successfully. >2014-09-10 12:27:11,869 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Start scheduling to enable vds load balancer >2014-09-10 12:27:11,877 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Finished scheduling to enable vds load balancer >2014-09-10 12:27:11,877 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Start HA Reservation check >2014-09-10 12:27:11,878 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Finished HA Reservation check >2014-09-10 12:27:11,889 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-10 12:27:11,907 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 49) Init VM custom properties utilities >2014-09-10 12:27:11,907 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 49) Init device custom properties utilities >2014-09-10 12:27:11,919 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Initializing Scheduling manager >2014-09-10 12:27:11,925 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 65 >2014-09-10 12:27:12,168 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) External scheduler disabled, discovery skipped >2014-09-10 12:27:12,168 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 49) Initialized Scheduling manager >2014-09-10 12:27:12,169 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 49) Initializing DWH Heart Beat >2014-09-10 12:27:12,172 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 49) DWH Heart Beat initialized >2014-09-10 12:27:13,767 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-2) Invalid locale found in configuration: >2014-09-10 12:27:13,767 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-2) Invalid locale found in configuration: >2014-09-10 12:27:15,778 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:15,784 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:15,786 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:15,789 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-7) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:15,836 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-7) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:15,837 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:15,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: UNASSIGNED not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-10 12:27:15,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-10 12:27:15,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-10 12:27:15,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-10 12:27:15,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STOPPED not exist in string table >2014-09-10 12:27:15,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_STARTED not exist in string table >2014-09-10 12:27:15,947 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: DWH_ERROR not exist in string table >2014-09-10 12:27:15,947 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,948 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-10 12:27:15,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-09-10 12:27:15,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-09-10 12:27:15,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-09-10 12:27:15,951 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-09-10 12:27:16,492 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:16,519 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:16,656 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Failed to refresh VDS , vds = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2014-09-10 12:27:19,732 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:19,733 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:19,733 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:19,734 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-15) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:19,736 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-15) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:19,737 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:19,765 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-15) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:19,783 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-15) Failed to refresh VDS , vds = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack, VDS Network Error, continuing. >java.net.ConnectException: Connection refused >2014-09-10 12:27:22,800 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:22,801 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:22,802 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:22,802 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-24) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:22,808 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-24) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:22,808 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:22,943 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-24) Server failed to respond, vds_id = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = java.net.ConnectException: Connection refused >2014-09-10 12:27:22,962 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-24) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-10 12:27:23,230 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-4) ResourceManager::vdsNotResponding entered for Host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 12:27:23,286 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-24) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:23,516 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginAdminUserCommand internal: false. >2014-09-10 12:27:23,704 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 12:27:23,764 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-4) [6236bbb6] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:27:23,772 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-4) [6236bbb6] Opening SSH Soft Fencing session on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 12:27:24,283 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-4) [6236bbb6] SSH Soft Fencing command failed on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com: SSH session closed during connection 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' >Stdout: >Stderr: >Stacktrace: java.io.IOException: SSH session closed during connection 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com': java.io.IOException: SSH session closed during connection 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.connect(SSHClient.java:309) [uutils.jar:] > at org.ovirt.engine.core.bll.utils.EngineSSHClient.connect(EngineSSHClient.java:59) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:87) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:243) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-10 12:27:24,475 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-4) [4992ab4d] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:27:24,475 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-4) [4992ab4d] Failed to run Fence script on vds:rack. >2014-09-10 12:27:24,514 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-4) [4992ab4d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-09-10 12:27:24,514 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-4) [4992ab4d] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-09-10 12:27:24,514 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-4) [4992ab4d] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:27:26,352 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:26,353 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:26,353 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:26,353 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-28) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:26,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-28) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:26,358 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-28) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:26,384 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-28) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:27,303 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-09-10 12:27:29,410 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:29,411 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:29,411 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:29,412 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-34) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:29,414 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-34) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:29,414 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:29,428 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:32,496 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:32,497 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:32,497 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:32,498 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-43) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:32,500 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-43) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:32,500 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-43) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:32,524 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:35,550 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:35,550 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:35,551 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:35,551 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-47) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:35,555 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-47) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:35,557 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-47) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:35,573 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-47) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:38,589 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:38,590 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:38,591 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:38,591 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-54) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:38,594 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-54) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:38,594 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-54) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:38,611 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-54) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:41,629 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:41,630 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:41,630 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:41,631 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-61) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:41,633 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-61) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:41,633 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-61) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:41,649 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-61) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:44,671 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:44,672 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:44,673 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:44,673 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-70) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:44,675 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-70) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:44,676 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-70) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:44,688 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-70) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:46,648 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-2) [1e2731f0] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:27:46,660 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-6-thread-15) [1e2731f0] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:27:46,686 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (org.ovirt.thread.pool-6-thread-15) [22cd7068] Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:27:46,697 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-6-thread-15) [22cd7068] START, UpdateVdsVMsClearedVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: 44748c >2014-09-10 12:27:46,716 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-6-thread-15) [22cd7068] FINISH, UpdateVdsVMsClearedVDSCommand, log id: 44748c >2014-09-10 12:27:46,746 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-15) [22cd7068] Correlation ID: 22cd7068, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host rack were changed to 'Down' by admin >2014-09-10 12:27:46,780 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-15) [22cd7068] Correlation ID: 1e2731f0, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host rack was started. >2014-09-10 12:27:46,781 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-6-thread-15) [22cd7068] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:27:47,713 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:47,717 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:47,717 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:47,717 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-76) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:47,720 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:47,720 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:47,748 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:50,777 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:50,778 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:50,779 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:50,779 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-80) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:50,784 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:50,784 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:50,810 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:53,834 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:53,835 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:53,835 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:53,836 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-90) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:53,838 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-90) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:53,838 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-90) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:53,860 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-90) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:56,881 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:56,882 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:56,882 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:56,883 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-94) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:56,888 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-94) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:56,888 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-94) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:56,917 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:59,943 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:27:59,944 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:59,945 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:59,945 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-100) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:27:59,947 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:27:59,948 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:27:59,960 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:02,973 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:28:02,974 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:02,974 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:02,975 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-10) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:02,977 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-10) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:28:02,977 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-10) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:28:02,992 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_65] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:06,023 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:28:06,024 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:06,024 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:06,024 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-14) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:06,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-14) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:28:06,041 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-14) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:28:06,053 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-14) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:09,067 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:28:09,067 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:09,068 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:09,068 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-21) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:175) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:09,071 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-21) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: java.net.ConnectException: Connection refused >2014-09-10 12:28:09,071 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-21) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:28:09,091 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-21) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.ConnectException: Connection refused > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:28:11,312 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-24) [cd923b] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:28:11,467 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-24) [cd923b] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:28:11,474 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-24) [cd923b] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 25adf1bf >2014-09-10 12:28:11,503 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-24) [cd923b] FINISH, SetVdsStatusVDSCommand, log id: 25adf1bf >2014-09-10 12:28:11,529 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-24) [cd923b] Correlation ID: cd923b, Job ID: 0e29fc1f-4548-487d-94bf-1061f25e104e, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-10 12:28:12,103 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-34) Initializing Host: rack >2014-09-10 12:28:22,518 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-10 12:28:22,519 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-10 12:28:22,519 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-9) [46799a9e] Running command: InstallVdsCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN >2014-09-10 12:28:22,669 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-10 12:28:22,669 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-9) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-10 12:28:22,677 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (ajp-/127.0.0.1:8702-9) [46799a9e] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 12:28:22,692 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:28:22,699 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Before Installation host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, rack >2014-09-10 12:28:22,702 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [46799a9e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 12:28:22,746 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-25) [46799a9e] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3abd3632 >2014-09-10 12:28:22,747 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [46799a9e] Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Host rack configuration was updated by admin. >2014-09-10 12:28:22,779 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-25) [46799a9e] FINISH, SetVdsStatusVDSCommand, log id: 3abd3632 >2014-09-10 12:28:22,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1 >2014-09-10 12:28:22,940 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1. >2014-09-10 12:28:22,984 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-10 12:28:23,006 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-25) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-10 12:28:24,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-10 12:28:24,093 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-10 12:28:24,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-10 12:28:24,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-10 12:28:24,144 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-10 12:28:24,158 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-10 12:28:31,295 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-10 12:28:31,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-10 12:28:31,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-10 12:28:31,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-10 12:28:31,727 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-10 12:28:31,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-10 12:28:31,762 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140910122823-iw36ig.log' >2014-09-10 12:28:31,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140910122823-iw36ig.log'. >2014-09-10 12:28:31,883 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-10 12:28:31,953 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 12:28:32,077 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-10 12:28:32,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-10 12:28:32,127 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-10 12:28:32,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-10 12:28:32,175 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-10 12:28:32,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-10 12:28:32,413 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-10 12:28:32,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-10 12:28:32,461 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 12:28:32,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 12:28:32,471 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-10 12:28:32,479 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-10 12:28:34,505 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 12:28:34,522 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 12:28:34,523 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-10 12:28:34,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-10 12:28:36,002 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-10 12:28:36,017 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-10 12:28:37,288 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-10 12:28:37,303 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-10 12:28:37,355 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-10 12:28:37,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-10 12:28:38,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-10 12:28:38,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-10 12:28:38,424 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-10 12:28:38,459 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-10 12:28:40,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-10 12:28:40,243 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-10 12:28:40,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910122840-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-46799a9e.log' >2014-09-10 12:28:40,280 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910122840-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-46799a9e.log'. >2014-09-10 12:28:40,527 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-10 12:28:40,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-10 12:28:40,665 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:28:41,038 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-25) [46799a9e] Engine managed to communicate with VDSM agent on host rack >2014-09-10 12:28:41,575 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-10 12:28:41,605 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] START, SetupNetworksVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=c97613e8-b860-495b-9207-7715533de9eb, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=ca117a0b-8f32-4c38-8b4c-aa15bd19f786, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 5637f29e >2014-09-10 12:28:41,619 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] FINISH, SetupNetworksVDSCommand, log id: 5637f29e >2014-09-10 12:30:44,135 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-09-10 12:30:44,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 12:30:44,808 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-11) [6fb0c3a2] Running command: LogoutBySessionCommand internal: false. >2014-09-10 12:30:44,831 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [515f938] Running command: LogoutUserCommand internal: false. >2014-09-10 12:30:44,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [515f938] Correlation ID: 515f938, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:31:06,839 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2014-09-10 12:31:06,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 12:31:06,876 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-9) [7cfa7bf9] Running command: LogoutBySessionCommand internal: false. >2014-09-10 12:31:06,881 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [2bdae329] Running command: LogoutUserCommand internal: false. >2014-09-10 12:31:06,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [2bdae329] Correlation ID: 2bdae329, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:31:20,849 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-10 12:31:20,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 12:31:22,473 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-4) [4d34c5fc] Running command: LogoutBySessionCommand internal: false. >2014-09-10 12:31:22,481 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [33e6a820] Running command: LogoutUserCommand internal: false. >2014-09-10 12:31:22,489 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [33e6a820] Correlation ID: 33e6a820, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:31:41,802 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:31:41,804 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Command SetupNetworksVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=c97613e8-b860-495b-9207-7715533de9eb, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=ca117a0b-8f32-4c38-8b4c-aa15bd19f786, vdsId=7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:31:41,804 ERROR [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Command org.ovirt.engine.core.bll.network.host.SetupNetworksCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' (Failed with error VDS_NETWORK_ERROR and code 5022) >2014-09-10 12:31:41,823 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Failed to configure management network: Failed to configure management network on host rack due to setup networks failure. >2014-09-10 12:31:41,828 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Host installation failed for host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, rack.: org.ovirt.engine.core.bll.VdsCommand$VdsInstallException: Failed to configure management network on the host > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:206) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:164) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:191) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.runCommands(MultipleActionsRunner.java:158) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$2.run(MultipleActionsRunner.java:167) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: org.ovirt.engine.core.bll.network.NetworkConfigurator$NetworkConfiguratorException: Failed to configure management network > at org.ovirt.engine.core.bll.network.NetworkConfigurator.configureManagementNetwork(NetworkConfigurator.java:255) [bll.jar:] > at org.ovirt.engine.core.bll.network.NetworkConfigurator.createManagementNetworkIfRequired(NetworkConfigurator.java:82) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.configureManagementNetwork(InstallVdsInternalCommand.java:198) [bll.jar:] > ... 18 more > >2014-09-10 12:31:41,832 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=NonOperational, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1c72fb92 >2014-09-10 12:31:41,852 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] FINISH, SetVdsStatusVDSCommand, log id: 1c72fb92 >2014-09-10 12:31:41,872 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Correlation ID: 46799a9e, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. Failed to configure management network on the host. >2014-09-10 12:31:41,872 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-25) [1be7dae4] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 12:35:00,041 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-50) Autorecovering 1 hosts >2014-09-10 12:35:00,041 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-50) Autorecovering hosts id: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, name : rack >2014-09-10 12:35:00,050 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 12:35:00,050 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:35:00,051 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-10 12:35:00,051 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-10 12:35:00,205 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6ad76cbd >2014-09-10 12:35:00,232 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] FINISH, SetVdsStatusVDSCommand, log id: 6ad76cbd >2014-09-10 12:35:00,251 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-10 12:35:00,288 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Correlation ID: 5b5b66dc, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-10 12:35:00,289 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-50) [5b5b66dc] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 12:35:00,731 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:35:00,735 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:252) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:295) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.read(SSLEngineNioHelper.java:51) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.read(SSLClient.java:81) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.readBuffer(ReactorClient.java:203) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.stomp.StompCommonClient.processIncoming(StompCommonClient.java:90) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.process(ReactorClient.java:144) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:115) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:38:02,936 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@5944c12d >2014-09-10 12:38:02,937 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) HostName = rack >2014-09-10 12:38:02,937 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-52) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:38:02,937 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:38:02,951 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:38:03,007 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-52) Server failed to respond, vds_id = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 120, error = VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:38:03,019 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-52) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-10 12:38:03,118 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-32) ResourceManager::vdsNotResponding entered for Host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 12:38:03,135 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-52) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-10 12:38:03,162 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-32) [62d854ff] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:38:03,162 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-32) [62d854ff] Failed to run Fence script on vds:rack. >2014-09-10 12:38:03,182 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-32) [62d854ff] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-09-10 12:38:03,182 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-32) [62d854ff] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-09-10 12:38:03,182 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-32) [62d854ff] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:38:06,163 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:38:06,164 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.Reactor] (SSL Stomp Reactor) Unable to process messages: java.nio.channels.ClosedChannelException > at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:265) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:474) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.write(SSLEngineNioHelper.java:104) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLEngineNioHelper.process(SSLEngineNioHelper.java:125) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.pendingOperations(SSLClient.java:74) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.SSLClient.process(SSLClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.processChannels(Reactor.java:86) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:62) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 12:41:08,502 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@14e1d12a >2014-09-10 12:41:08,503 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) HostName = rack >2014-09-10 12:41:08,503 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-16) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:41:08,503 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-16) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:41:08,515 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:41:08,521 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-16) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-10 12:41:11,545 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:41:14,676 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (ajp-/127.0.0.1:8702-1) [59fa8473] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:41:14,677 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-6-thread-35) [59fa8473] Running command: FenceVdsManualyCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:41:14,686 INFO [org.ovirt.engine.core.bll.ClearNonResponsiveVdsVmsCommand] (org.ovirt.thread.pool-6-thread-35) [49d242c4] Running command: ClearNonResponsiveVdsVmsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:41:14,692 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-6-thread-35) [49d242c4] START, UpdateVdsVMsClearedVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: 23d936e7 >2014-09-10 12:41:28,434 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-36) [39b96113] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:41:28,481 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-36) [39b96113] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:41:28,487 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [39b96113] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1757e050 >2014-09-10 12:44:13,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand return value > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturnForXmlRpc@91bf26 >2014-09-10 12:44:13,574 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) HostName = rack >2014-09-10 12:44:13,574 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-79) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' >2014-09-10 12:44:13,574 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-79) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:44:13,588 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-79) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-10 12:44:13,595 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-79) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:16) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-10 12:44:13,631 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [39b96113] FINISH, SetVdsStatusVDSCommand, log id: 1757e050 >2014-09-10 12:44:13,646 INFO [org.ovirt.engine.core.vdsbroker.UpdateVdsVMsClearedVDSCommand] (org.ovirt.thread.pool-6-thread-35) [49d242c4] FINISH, UpdateVdsVMsClearedVDSCommand, log id: 23d936e7 >2014-09-10 12:44:13,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [39b96113] Correlation ID: 39b96113, Job ID: 34b4e15b-e544-4bdc-afd3-6367282835af, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-10 12:44:13,661 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [49d242c4] Correlation ID: 49d242c4, Call Stack: null, Custom Event ID: -1, Message: All VMs' status on Non Responsive Host rack were changed to 'Down' by admin >2014-09-10 12:44:13,702 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [49d242c4] Correlation ID: 59fa8473, Call Stack: null, Custom Event ID: -1, Message: Manual fence for host rack was started. >2014-09-10 12:44:13,703 INFO [org.ovirt.engine.core.bll.storage.FenceVdsManualyCommand] (org.ovirt.thread.pool-6-thread-35) [49d242c4] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS_FENCE >, sharedLocks= ] >2014-09-10 12:58:35,474 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-38) [3bff881a] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 12:58:35,527 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-38) [3bff881a] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:58:35,534 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-38) [3bff881a] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5f143ac >2014-09-10 12:58:48,564 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-44) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:82) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:58:48,564 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-46) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:58:53,574 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-59) Failed to invoke scheduled method refreshLightWeightData: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor61.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterSyncJob.refreshLightWeightData(GlusterSyncJob.java:82) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:58:53,573 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-56) Failed to invoke scheduled method refreshGlusterServices: java.lang.reflect.InvocationTargetException > at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:573) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] > at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.1.RELEASE] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:141) [dal.jar:] > at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:135) [dal.jar:] > at org.ovirt.engine.core.dao.VdsGroupDAODbFacadeImpl.getAll(VdsGroupDAODbFacadeImpl.java:128) [dal.jar:] > at org.ovirt.engine.core.bll.gluster.GlusterServiceSyncJob.refreshGlusterServices(GlusterServiceSyncJob.java:54) [bll.jar:] > ... 6 more >Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) > at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) [spring-jdbc.jar:3.1.1.RELEASE] > at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) [spring-jdbc.jar:3.1.1.RELEASE] > ... 18 more >Caused by: javax.resource.ResourceException: IJ000451: The connection manager is shutdown: java:/ENGINEDataSource > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.getManagedConnection(AbstractConnectionManager.java:345) > at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.getManagedConnection(TxConnectionManagerImpl.java:368) > at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:488) > at org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:143) > ... 20 more > >2014-09-10 12:59:04,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-10 12:59:04,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-10 12:59:04,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-10 12:59:04,520 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-10 12:59:04,520 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-10 12:59:04,520 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-10 12:59:04,520 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-10 12:59:04,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-10 12:59:04,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-10 12:59:04,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-10 12:59:04,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-10 12:59:04,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-10 12:59:04,533 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-10 12:59:04,534 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_UI" is "/var/lib/ovirt-engine/reports.xml". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-10 12:59:04,535 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-10 12:59:04,543 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-10 12:59:04,544 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-10 12:59:13,281 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 49) Start initializing Backend >2014-09-10 12:59:13,790 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 49) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-10 12:59:13,933 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 49) Running ovirt-engine 3.5.0-0.10.master.el6ev >2014-09-10 12:59:13,933 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 49) Start initializing dictionaries >2014-09-10 12:59:13,935 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 49) Finished initializing dictionaries >2014-09-10 12:59:13,935 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Start initializing AuditLogCleanupManager >2014-09-10 12:59:13,935 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-10 12:59:13,958 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Finished initializing AuditLogCleanupManager >2014-09-10 12:59:13,959 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Start initializing CommandEntityCleanupManager >2014-09-10 12:59:13,959 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-10 12:59:13,961 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 49) Finished initializing CommandEntityCleanupManager >2014-09-10 12:59:13,963 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 49) Start initializing TagsDirector >2014-09-10 12:59:13,973 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 49) Tag root added to tree >2014-09-10 12:59:13,990 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 49) Finished initializing TagsDirector >2014-09-10 12:59:13,990 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 49) Start initializing IsoDomainListSyncronizer >2014-09-10 12:59:13,996 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 49) Finished initializing IsoDomainListSyncronizer >2014-09-10 12:59:14,032 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 49) Completed initializing handlers >2014-09-10 12:59:14,060 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 49) Start initializing ErrorTranslatorImpl >2014-09-10 12:59:14,073 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 49) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-10 12:59:14,074 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 49) Finished initializing ErrorTranslatorImpl >2014-09-10 12:59:14,075 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 49) Start initializing ErrorTranslatorImpl >2014-09-10 12:59:14,075 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 49) Finished initializing ErrorTranslatorImpl >2014-09-10 12:59:14,075 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 49) Mark incomplete jobs as UNKNOWN >2014-09-10 12:59:14,121 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 49) Start initializing JobRepositoryCleanupManager >2014-09-10 12:59:14,121 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 49) Finished initializing JobRepositoryCleanupManager >2014-09-10 12:59:14,122 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 49) Start initializing AutoRecoveryManager >2014-09-10 12:59:14,123 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 49) Finished initializing AutoRecoveryManager >2014-09-10 12:59:14,125 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 49) Start initializing ExecutionMessageDirector >2014-09-10 12:59:14,129 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 49) Finished initializing ExecutionMessageDirector >2014-09-10 12:59:14,252 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Loading extension 'builtin-authn-internal' >2014-09-10 12:59:14,253 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Extension 'builtin-authn-internal' loaded >2014-09-10 12:59:14,255 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Loading extension 'internal' >2014-09-10 12:59:14,257 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Extension 'internal' loaded >2014-09-10 12:59:14,257 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Initializing extension 'builtin-authn-internal' >2014-09-10 12:59:14,257 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Extension 'builtin-authn-internal' initialized >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Initializing extension 'internal' >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Extension 'internal' initialized >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Start of enabled extensions list >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-10 12:59:14,258 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 49) End of enabled extensions list >2014-09-10 12:59:14,434 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 54) Start initializing DbUserCacheManager >2014-09-10 12:59:14,446 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 54) Finished initializing DbUserCacheManager >2014-09-10 12:59:14,496 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 54) Initialization of AsyncTaskManager completed successfully. >2014-09-10 12:59:14,498 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) Start initializing ResourceManager >2014-09-10 12:59:14,599 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-10 12:59:14,817 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 54) Entered VdsManager constructor >2014-09-10 12:59:14,921 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 54) Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-10 12:59:15,118 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) VDS 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 was added to the Resource Manager >2014-09-10 12:59:15,171 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) Finished initializing ResourceManager >2014-09-10 12:59:15,178 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 54) Initialization of OvfDataUpdater completed successfully. >2014-09-10 12:59:15,179 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Start scheduling to enable vds load balancer >2014-09-10 12:59:15,179 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Finished scheduling to enable vds load balancer >2014-09-10 12:59:15,179 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Start HA Reservation check >2014-09-10 12:59:15,181 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Finished HA Reservation check >2014-09-10 12:59:15,186 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-10 12:59:15,201 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 54) Init VM custom properties utilities >2014-09-10 12:59:15,202 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 54) Init device custom properties utilities >2014-09-10 12:59:15,213 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Initializing Scheduling manager >2014-09-10 12:59:15,253 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 65 >2014-09-10 12:59:15,297 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) External scheduler disabled, discovery skipped >2014-09-10 12:59:15,297 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Initialized Scheduling manager >2014-09-10 12:59:15,298 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 54) Initializing DWH Heart Beat >2014-09-10 12:59:15,299 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 54) DWH Heart Beat initialized >2014-09-10 12:59:18,360 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 12:59:18,922 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]), log id: 737412bd >2014-09-10 12:59:19,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 737412bd >2014-09-10 12:59:19,467 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: UNASSIGNED not exist in string table >2014-09-10 12:59:19,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: VDS_AUTO_FENCE_STATUS not exist in string table >2014-09-10 12:59:19,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: VDS_AUTO_FENCE_STATUS_FAILED not exist in string table >2014-09-10 12:59:19,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: VDS_AUTO_FENCE_FAILED_CALL_FENCE_SPM not exist in string table >2014-09-10 12:59:19,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-10 12:59:19,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-10 12:59:19,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-10 12:59:19,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: DWH_STOPPED not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: DWH_STARTED not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: DWH_ERROR not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_REMOVE_AUDIT_LOG not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_REMOVE_AUDIT_LOG_FAILED not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG not exist in string table >2014-09-10 12:59:19,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] AuditLogType: USER_CLEAR_ALL_DISMISSED_AUDIT_LOG_FAILED not exist in string table >2014-09-10 12:59:19,807 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [34014e32] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:19,814 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [34014e32] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 731862af >2014-09-10 12:59:19,862 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [34014e32] FINISH, SetVdsStatusVDSCommand, log id: 731862af >2014-09-10 12:59:19,926 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [34014e32] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-10 12:59:20,481 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [34014e32] Correlation ID: 34014e32, Job ID: 7c21bda2-6400-41ce-a95f-e545b60e9006, Call Stack: null, Custom Event ID: -1, Message: Host rack does not comply with the cluster Default networks, the following networks are missing on host: 'rhevm' >2014-09-10 12:59:20,531 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [7041b420] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:20,605 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [7041b420] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]), log id: 2698add4 >2014-09-10 12:59:20,610 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) [7041b420] FINISH, GetHardwareInfoVDSCommand, log id: 2698add4 >2014-09-10 12:59:20,933 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [6acc6dc] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:20,938 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [6acc6dc] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 6e948eb3 >2014-09-10 12:59:20,961 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-6) [6acc6dc] FINISH, SetVdsStatusVDSCommand, log id: 6e948eb3 >2014-09-10 12:59:20,969 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-6) [6acc6dc] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-10 12:59:20,998 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [16ec4930] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:21,022 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) [16ec4930] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-10 12:59:21,276 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) [16ec4930] Initializing Host: rack >2014-09-10 12:59:21,302 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-6) [1a0445c6] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:21,316 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [54660788] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 12:59:21,327 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) [54660788] Host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-10 12:59:33,380 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-09-10 12:59:33,406 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 12:59:40,227 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [51667022] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,238 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [26d7db08] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,240 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [23cc4a01] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,243 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [20c626ed] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,265 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [31c37a79] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,327 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [63537fc9] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,523 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [26d7db08] Correlation ID: 26d7db08, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [51667022] Correlation ID: 51667022, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,548 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [23cc4a01] Correlation ID: 23cc4a01, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [20c626ed] Correlation ID: 20c626ed, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [31c37a79] Correlation ID: 31c37a79, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,585 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [63537fc9] Correlation ID: 63537fc9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,634 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [55c76991] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,652 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-10 12:59:40,665 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-1) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-10 12:59:40,670 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [132af7bc] Running command: LogoutUserCommand internal: false. >2014-09-10 12:59:40,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [55c76991] Correlation ID: 55c76991, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:40,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [132af7bc] Correlation ID: 132af7bc, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 12:59:41,003 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-10 12:59:41,003 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-10 12:59:55,264 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginAdminUserCommand internal: false. >2014-09-10 12:59:55,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 13:00:00,125 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-93) Autorecovering 1 hosts >2014-09-10 13:00:00,125 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-93) Autorecovering hosts id: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, name : rack >2014-09-10 13:00:00,135 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 13:00:00,136 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Running command: ActivateVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 13:00:00,136 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-10 13:00:00,136 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-10 13:00:00,284 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 12dbc0c2 >2014-09-10 13:00:00,328 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] FINISH, SetVdsStatusVDSCommand, log id: 12dbc0c2 >2014-09-10 13:00:00,561 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-10 13:00:00,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-93) [92dad18] Correlation ID: 92dad18, Call Stack: null, Custom Event ID: -1, Message: Host rack was autorecovered. >2014-09-10 13:00:00,607 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (DefaultQuartzScheduler_Worker-93) [92dad18] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 13:00:00,786 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-96) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, vds=Host[rack,7ebdb254-efb6-480b-8c4e-a7dbdc759ed4]), log id: 65ae2ff0 >2014-09-10 13:00:00,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-96) FINISH, GetHardwareInfoVDSCommand, log id: 65ae2ff0 >2014-09-10 13:00:01,105 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-96) [67050309] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 13:00:01,110 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-96) [67050309] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=NonOperational, nonOperationalReason=NETWORK_UNREACHABLE, stopSpmFailureLogged=false), log id: 4ac4c6c7 >2014-09-10 13:00:01,155 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-96) [67050309] FINISH, SetVdsStatusVDSCommand, log id: 4ac4c6c7 >2014-09-10 13:00:01,162 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-96) [67050309] Host rack is set to Non-Operational, it is missing the following networks: rhevm >2014-09-10 13:00:01,223 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-96) [3b8ca6e0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 13:00:01,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [3b8ca6e0] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to NonOperational. >2014-09-10 13:00:01,354 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-96) [21e034cd] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 13:00:01,360 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) [21e034cd] Host 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack is already in NonOperational status for reason NETWORK_UNREACHABLE. SetNonOperationalVds command is skipped. >2014-09-10 13:00:24,594 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-9) [6e736f53] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 13:00:24,612 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [6e736f53] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 53d16b1c >2014-09-10 13:00:24,663 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [6e736f53] FINISH, SetVdsStatusVDSCommand, log id: 53d16b1c >2014-09-10 13:00:24,958 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-9) [6e736f53] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDS >2014-09-10 13:00:24,974 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-9) [6e736f53] Correlation ID: 6e736f53, Job ID: ed1d360e-de71-4da3-84f6-bece0777a159, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-10 13:00:25,529 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-56) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 : rack >2014-09-10 13:00:25,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-11) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-10 13:00:25,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-11) Removing vds [7ebdb254-efb6-480b-8c4e-a7dbdc759ed4] from the domain in maintenance cache >2014-09-10 13:00:25,660 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-11) Removing host(s) [7ebdb254-efb6-480b-8c4e-a7dbdc759ed4] from hosts unseen domain report cache >2014-09-10 13:00:36,175 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [60fa1c6d] Lock Acquired to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 13:00:36,229 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-10 13:00:36,357 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] START, RemoveVdsVDSCommand( HostId = 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4), log id: 69101be1 >2014-09-10 13:00:36,357 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] vdsManager::disposing >2014-09-10 13:00:36,362 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] FINISH, RemoveVdsVDSCommand, log id: 69101be1 >2014-09-10 13:00:36,373 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] Correlation ID: 60fa1c6d, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-10 13:00:36,373 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-13) [60fa1c6d] Lock freed to object EngineLock [exclusiveLocks= key: 7ebdb254-efb6-480b-8c4e-a7dbdc759ed4 value: VDS >, sharedLocks= ] >2014-09-10 13:00:58,564 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-4) [523871a6] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-10 13:00:58,753 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:00:58,818 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-10 13:00:58,822 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] START, RemoveVdsVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc), log id: 1aa5cd79 >2014-09-10 13:00:58,824 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] FINISH, RemoveVdsVDSCommand, log id: 1aa5cd79 >2014-09-10 13:00:58,828 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] START, AddVdsVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc), log id: 58229403 >2014-09-10 13:00:58,829 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] AddVds - entered , starting logic to add VDS 62d62bcc-dea3-4792-b313-9dcd4da8d3dc >2014-09-10 13:00:58,831 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] AddVds - VDS 62d62bcc-dea3-4792-b313-9dcd4da8d3dc was added, will try to add it to the resource manager >2014-09-10 13:00:58,831 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [349d74fb] Entered VdsManager constructor >2014-09-10 13:00:58,848 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-4) [349d74fb] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-10 13:00:59,018 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-4) [349d74fb] VDS 62d62bcc-dea3-4792-b313-9dcd4da8d3dc was added to the Resource Manager >2014-09-10 13:00:59,019 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-4) [349d74fb] FINISH, AddVdsVDSCommand, log id: 58229403 >2014-09-10 13:00:59,039 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [349d74fb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 13:00:59,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [349d74fb] Correlation ID: 523871a6, Job ID: bdfdc66c-d14e-4d7a-9e9f-fe7e50b4f8d0, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-10 13:00:59,115 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Lock Acquired to object EngineLock [exclusiveLocks= key: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc value: VDS >, sharedLocks= ] >2014-09-10 13:00:59,139 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:00:59,148 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Before Installation host 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, rack >2014-09-10 13:00:59,159 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6cf90d7d >2014-09-10 13:00:59,170 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] FINISH, SetVdsStatusVDSCommand, log id: 6cf90d7d >2014-09-10 13:00:59,220 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1 >2014-09-10 13:00:59,247 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1. >2014-09-10 13:00:59,266 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-10 13:00:59,278 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-14) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-10 13:01:00,391 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-10 13:01:00,407 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-10 13:01:00,419 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-10 13:01:00,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-10 13:01:00,434 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-10 13:01:00,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-10 13:01:02,023 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-26) Initializing Host: rack >2014-09-10 13:01:11,375 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-10 13:01:11,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-10 13:01:11,390 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-10 13:01:11,406 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-10 13:01:11,773 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-10 13:01:11,787 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-10 13:01:11,806 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140910130059-bknidl.log' >2014-09-10 13:01:11,828 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140910130059-bknidl.log'. >2014-09-10 13:01:11,880 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-10 13:01:11,911 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 13:01:12,039 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-10 13:01:12,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-10 13:01:12,086 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-10 13:01:12,107 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-10 13:01:12,125 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-10 13:01:12,144 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-10 13:01:12,370 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-10 13:01:12,384 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-10 13:01:12,423 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 13:01:12,433 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 13:01:12,433 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-10 13:01:12,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-10 13:01:19,691 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 13:01:19,707 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 13:01:19,707 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-10 13:01:19,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-10 13:01:21,198 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-10 13:01:21,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-10 13:01:22,457 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-10 13:01:22,486 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-10 13:01:22,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-10 13:01:22,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-10 13:01:23,293 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-10 13:01:23,310 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-10 13:01:23,567 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-10 13:01:23,581 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-10 13:01:25,368 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-10 13:01:25,383 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-10 13:01:25,398 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910130125-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-6fbdab31.log' >2014-09-10 13:01:25,416 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910130125-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-6fbdab31.log'. >2014-09-10 13:01:25,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-10 13:01:25,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6fbdab31, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-10 13:01:26,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSRecoveringException: Recovering from crash or Initializing >2014-09-10 13:01:26,042 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Command PollVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc) execution failed. Exception: VDSRecoveringException: Recovering from crash or Initializing >2014-09-10 13:01:26,588 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-14) [6fbdab31] Engine managed to communicate with VDSM agent on host rack >2014-09-10 13:01:27,331 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-10 13:01:27,338 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] START, SetupNetworksVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=2ae645fb-b95f-4ac7-bf44-d6ea67f22157, vdsId=62d62bcc-dea3-4792-b313-9dcd4da8d3dc, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=bab47b56-1de3-4e38-b789-a162c716830b, vdsId=62d62bcc-dea3-4792-b313-9dcd4da8d3dc, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 16957cae >2014-09-10 13:01:27,343 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] FINISH, SetupNetworksVDSCommand, log id: 16957cae >2014-09-10 13:01:30,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Command PollVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-09-10 13:01:30,924 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-09-10 13:01:33,932 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Command PollVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-09-10 13:01:33,932 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-09-10 13:01:36,936 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Command PollVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-09-10 13:01:36,937 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-09-10 13:01:39,946 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Command PollVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc) execution failed. Exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call >2014-09-10 13:01:39,946 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2ef40273] Timeout waiting for VDSM response. java.util.concurrent.TimeoutException >2014-09-10 13:01:41,976 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-10 13:01:41,979 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] START, SetSafeNetworkConfigVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc), log id: 1cff268c >2014-09-10 13:01:42,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] FINISH, SetSafeNetworkConfigVDSCommand, log id: 1cff268c >2014-09-10 13:01:42,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [72bc084] Correlation ID: 72bc084, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host rack >2014-09-10 13:01:42,058 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 455dae10 >2014-09-10 13:01:42,088 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] FINISH, SetVdsStatusVDSCommand, log id: 455dae10 >2014-09-10 13:01:42,088 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] After Installation host rack, VDS >2014-09-10 13:01:42,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [72bc084] Correlation ID: 6fbdab31, Job ID: bdfdc66c-d14e-4d7a-9e9f-fe7e50b4f8d0, Call Stack: null, Custom Event ID: -1, Message: Host rack installed >2014-09-10 13:01:42,123 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [72bc084] Lock freed to object EngineLock [exclusiveLocks= key: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc value: VDS >, sharedLocks= ] >2014-09-10 13:01:44,413 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, vds=Host[rack,62d62bcc-dea3-4792-b313-9dcd4da8d3dc]), log id: 6fc4d38e >2014-09-10 13:01:44,492 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) FINISH, GetHardwareInfoVDSCommand, log id: 6fc4d38e >2014-09-10 13:01:44,540 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-20) [53caa8d2] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:01:44,889 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) [53caa8d2] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, vds=Host[rack,62d62bcc-dea3-4792-b313-9dcd4da8d3dc]), log id: 3b551c82 >2014-09-10 13:01:44,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-20) [53caa8d2] FINISH, GetHardwareInfoVDSCommand, log id: 3b551c82 >2014-09-10 13:01:45,093 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-20) [6c9eb5a6] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-10 13:01:45,139 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-20) [6c9eb5a6] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc), log id: 76b4dea0 >2014-09-10 13:01:45,278 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-20) [6c9eb5a6] FINISH, SetMOMPolicyParametersVDSCommand, log id: 76b4dea0 >2014-09-10 13:01:45,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6c9eb5a6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-10 13:01:45,324 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [6c9eb5a6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 13:01:45,391 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-20) [30a2fdfe] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:01:45,413 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-20) [232c3800] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:22:55,293 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-41) [7469b73c] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 13:22:55,303 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-41) [7469b73c] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 50b027ab >2014-09-10 13:22:55,333 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-41) [7469b73c] FINISH, SetVdsStatusVDSCommand, log id: 50b027ab >2014-09-10 13:22:55,434 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-41) [7469b73c] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDS >2014-09-10 13:22:55,451 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-41) [7469b73c] Correlation ID: 7469b73c, Job ID: a6a2da1a-d1a5-47b9-82e6-bebfa3cb288f, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-10 13:22:58,393 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-92) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc : rack >2014-09-10 13:22:58,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-45) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-10 13:22:58,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-45) Removing vds [62d62bcc-dea3-4792-b313-9dcd4da8d3dc] from the domain in maintenance cache >2014-09-10 13:22:58,436 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-45) Removing host(s) [62d62bcc-dea3-4792-b313-9dcd4da8d3dc] from hosts unseen domain report cache >2014-09-10 13:23:18,768 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-10) [5ce283d7] Lock Acquired to object EngineLock [exclusiveLocks= key: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc value: VDS >, sharedLocks= ] >2014-09-10 13:23:18,780 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-10 13:23:18,836 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] START, RemoveVdsVDSCommand( HostId = 62d62bcc-dea3-4792-b313-9dcd4da8d3dc), log id: 5633cab9 >2014-09-10 13:23:18,836 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] vdsManager::disposing >2014-09-10 13:23:18,838 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] FINISH, RemoveVdsVDSCommand, log id: 5633cab9 >2014-09-10 13:23:18,847 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] Correlation ID: 5ce283d7, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-10 13:23:18,847 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-47) [5ce283d7] Lock freed to object EngineLock [exclusiveLocks= key: 62d62bcc-dea3-4792-b313-9dcd4da8d3dc value: VDS >, sharedLocks= ] >2014-09-10 13:23:51,765 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-1) [68a704d0] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-10 13:23:51,825 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 13:23:51,843 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-10 13:23:51,847 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] START, RemoveVdsVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 75f2fae4 >2014-09-10 13:23:51,847 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] FINISH, RemoveVdsVDSCommand, log id: 75f2fae4 >2014-09-10 13:23:51,849 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] START, AddVdsVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 7eecc7e3 >2014-09-10 13:23:51,849 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] AddVds - entered , starting logic to add VDS f689c28a-07de-4408-ac86-81382b6dd288 >2014-09-10 13:23:51,851 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] AddVds - VDS f689c28a-07de-4408-ac86-81382b6dd288 was added, will try to add it to the resource manager >2014-09-10 13:23:51,851 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-1) [41a5b178] Entered VdsManager constructor >2014-09-10 13:23:51,865 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-1) [41a5b178] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-10 13:23:51,866 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-1) [41a5b178] VDS f689c28a-07de-4408-ac86-81382b6dd288 was added to the Resource Manager >2014-09-10 13:23:51,866 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-1) [41a5b178] FINISH, AddVdsVDSCommand, log id: 7eecc7e3 >2014-09-10 13:23:51,879 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [41a5b178] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 13:23:51,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [41a5b178] Correlation ID: 68a704d0, Job ID: f04c0777-cd59-4cea-8771-d9946f27e6e2, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-10 13:23:51,916 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Lock Acquired to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-10 13:23:51,931 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 13:23:51,931 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Before Installation host f689c28a-07de-4408-ac86-81382b6dd288, rack >2014-09-10 13:23:51,936 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 75aee44b >2014-09-10 13:23:51,956 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] FINISH, SetVdsStatusVDSCommand, log id: 75aee44b >2014-09-10 13:23:51,983 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1 >2014-09-10 13:23:51,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 25:12:7b:de:30:1d:b3:5e:bf:d2:a6:c7:37:77:d3:b1. >2014-09-10 13:23:52,062 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-10 13:23:52,076 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-49) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-10 13:23:52,849 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-10 13:23:52,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-10 13:23:52,871 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-10 13:23:52,881 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-10 13:23:52,881 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-10 13:23:52,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-10 13:23:54,869 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-12) Initializing Host: rack >2014-09-10 13:23:58,115 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-10 13:23:58,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-10 13:23:58,127 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-10 13:23:58,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-10 13:23:58,512 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-10 13:23:58,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-10 13:23:58,534 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140910132351-ohur1q.log' >2014-09-10 13:23:58,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140910132351-ohur1q.log'. >2014-09-10 13:23:58,637 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-10 13:23:58,640 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 13:23:58,731 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-10 13:23:58,748 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-10 13:23:58,768 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-10 13:23:58,778 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-10 13:23:58,807 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-10 13:23:58,817 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-10 13:23:59,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-10 13:23:59,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-10 13:23:59,099 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 13:23:59,109 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 13:23:59,109 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-10 13:23:59,117 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-10 13:24:06,367 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-10 13:24:06,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-10 13:24:06,379 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-10 13:24:06,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-10 13:24:07,985 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-10 13:24:07,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-10 13:24:09,212 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-10 13:24:09,222 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-10 13:24:09,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-10 13:24:09,297 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-10 13:24:10,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-10 13:24:10,076 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-10 13:24:10,327 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-10 13:24:10,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-10 13:24:12,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-10 13:24:12,145 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-10 13:24:12,162 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910132412-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-5d003ac8.log' >2014-09-10 13:24:12,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140910132412-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-5d003ac8.log'. >2014-09-10 13:24:12,407 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-10 13:24:12,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 5d003ac8, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-10 13:24:12,501 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 13:24:12,502 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 13:24:12,505 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 13:24:12,506 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-6-thread-48) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:98) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:56) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:57) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:871) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$2.call(JsonRpcVdsServer.java:867) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:87) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:77) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-10 13:24:12,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-10 13:24:12,520 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Command PollVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-10 13:24:13,021 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 13:24:13,105 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Engine managed to communicate with VDSM agent on host rack >2014-09-10 13:24:13,433 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] The management network rhevm is already configured on host rack >2014-09-10 13:24:13,436 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 60d8558a >2014-09-10 13:24:13,455 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] FINISH, SetVdsStatusVDSCommand, log id: 60d8558a >2014-09-10 13:24:13,455 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] After Installation host rack, VDS >2014-09-10 13:24:13,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Correlation ID: 5d003ac8, Job ID: f04c0777-cd59-4cea-8771-d9946f27e6e2, Call Stack: null, Custom Event ID: -1, Message: Host rack installed >2014-09-10 13:24:13,481 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-49) [5d003ac8] Lock freed to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-10 13:24:15,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288]), log id: 57bc2aa8 >2014-09-10 13:24:15,978 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GetHardwareInfoVDSCommand, log id: 57bc2aa8 >2014-09-10 13:24:16,031 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-64) [488773a4] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 13:24:16,094 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) [488773a4] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288]), log id: 13f9a5c4 >2014-09-10 13:24:16,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) [488773a4] FINISH, GetHardwareInfoVDSCommand, log id: 13f9a5c4 >2014-09-10 13:24:16,151 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-64) [3d96d9a1] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-10 13:24:16,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-64) [3d96d9a1] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 7cfa360f >2014-09-10 13:24:16,225 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-64) [3d96d9a1] FINISH, SetMOMPolicyParametersVDSCommand, log id: 7cfa360f >2014-09-10 13:24:16,240 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) [3d96d9a1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-10 13:24:16,251 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) [3d96d9a1] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 13:24:16,282 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-64) [762ceea8] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 13:24:16,290 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-64) [29539bf3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 13:59:14,659 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-10 13:59:15,181 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-09-10 14:11:03,803 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [37e7e802] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,846 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-10) [295dc995] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,852 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [757a28c7] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [37e7e802] Correlation ID: 37e7e802, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:03,851 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [e16b7c2] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,859 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [2ac5a46a] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,888 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [2d2da368] Running command: LogoutUserCommand internal: false. >2014-09-10 14:11:03,890 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [e16b7c2] Correlation ID: e16b7c2, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:03,898 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [757a28c7] Correlation ID: 757a28c7, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:03,917 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [295dc995] Correlation ID: 295dc995, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:03,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [2ac5a46a] Correlation ID: 2ac5a46a, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:03,931 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [2d2da368] Correlation ID: 2d2da368, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 14:11:04,527 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-10 14:11:04,527 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-10 14:11:51,966 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginAdminUserCommand internal: false. >2014-09-10 14:11:51,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 14:13:44,829 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-31) [655aab1e] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 14:13:44,843 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-31) [655aab1e] START, SetVdsStatusVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 5b483271 >2014-09-10 14:13:44,861 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-31) [655aab1e] FINISH, SetVdsStatusVDSCommand, log id: 5b483271 >2014-09-10 14:13:44,956 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-31) [655aab1e] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 14:13:44,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-31) [655aab1e] Correlation ID: 655aab1e, Job ID: a757f9e0-ce1b-4af0-b821-875ea8d06b70, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-10 14:13:45,519 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = f689c28a-07de-4408-ac86-81382b6dd288 : rack >2014-09-10 14:13:45,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-33) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-10 14:13:45,554 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-33) Removing vds [f689c28a-07de-4408-ac86-81382b6dd288] from the domain in maintenance cache >2014-09-10 14:13:45,554 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-33) Removing host(s) [f689c28a-07de-4408-ac86-81382b6dd288] from hosts unseen domain report cache >2014-09-10 14:14:12,251 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-5) [7f7de479] Lock Acquired to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-10 14:14:12,286 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Running command: ActivateVdsCommand internal: false. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-10 14:14:12,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-10 14:14:12,287 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-10 14:14:12,301 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] START, SetVdsStatusVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3465ccf5 >2014-09-10 14:14:12,316 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] FINISH, SetVdsStatusVDSCommand, log id: 3465ccf5 >2014-09-10 14:14:12,327 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-10 14:14:12,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Correlation ID: 7f7de479, Job ID: 96539c7c-b0f9-4d55-8e62-71dd6aa9e2ba, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-10 14:14:12,347 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7f7de479] Lock freed to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-10 14:14:12,653 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288]), log id: 214c5d7a >2014-09-10 14:14:12,665 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, GetHardwareInfoVDSCommand, log id: 214c5d7a >2014-09-10 14:14:12,730 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-30) [71b56914] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 14:14:12,743 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-30) [7cadb7b2] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-10 14:14:12,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [7cadb7b2] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 3e156850 >2014-09-10 14:14:12,816 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-30) [7cadb7b2] FINISH, SetMOMPolicyParametersVDSCommand, log id: 3e156850 >2014-09-10 14:14:12,834 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [7cadb7b2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-10 14:14:12,847 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) [7cadb7b2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-10 14:14:12,878 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-30) [93c1504] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-10 14:48:37,260 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-09-10 14:48:37,270 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 14:59:14,676 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Principal admin::internal synchronized >2014-09-10 14:59:15,194 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-09-10 15:59:14,690 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Principal admin::internal synchronized >2014-09-10 15:59:15,198 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-10 16:59:14,703 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-80) Principal admin::internal synchronized >2014-09-10 16:59:15,201 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-10 17:59:14,717 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-21) Principal admin::internal synchronized >2014-09-10 17:59:15,205 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) Attempting to update VMs/Templates Ovf. >2014-09-10 18:43:03,051 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-10) [692cfcea] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35 value: STORAGE_CONNECTION >, sharedLocks= ] >2014-09-10 18:43:03,105 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-10) [692cfcea] Running command: AddStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:03,130 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-10) [692cfcea] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 56ee9a34 >2014-09-10 18:43:03,429 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-10) [692cfcea] FINISH, ConnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=0}, log id: 56ee9a34 >2014-09-10 18:43:03,464 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-10) [692cfcea] Lock freed to object EngineLock [exclusiveLocks= key: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35 value: STORAGE_CONNECTION >, sharedLocks= ] >2014-09-10 18:43:03,649 INFO [org.ovirt.engine.core.bll.storage.AddNFSStorageDomainCommand] (ajp-/127.0.0.1:8702-3) [2c076442] Running command: AddNFSStorageDomainCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:03,715 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 57d565e9 >2014-09-10 18:43:03,727 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 57d565e9 >2014-09-10 18:43:03,731 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] START, CreateStorageDomainVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storageDomain=org.ovirt.engine.core.common.businessentities.StorageDomainStatic@d75a0250, args=10.34.63.202:/mnt/export/nfs/lv3/pbenas/35), log id: 7e4d3e3f >2014-09-10 18:43:04,623 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] FINISH, CreateStorageDomainVDSCommand, log id: 7e4d3e3f >2014-09-10 18:43:04,628 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] START, GetStorageDomainStatsVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storageDomainId=eb633636-3019-47a9-9f5a-fc81f8dfd7dc), log id: 26008aca >2014-09-10 18:43:04,643 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-3) [2c076442] FINISH, GetStorageDomainStatsVDSCommand, return: org.ovirt.engine.core.common.businessentities.StorageDomain@123dee6d, log id: 26008aca >2014-09-10 18:43:04,739 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [2c076442] Correlation ID: 2c076442, Job ID: 35fcf1d2-d2df-4af9-af5e-812a6cb56304, Call Stack: null, Custom Event ID: -1, Message: Storage Domain master was added by admin >2014-09-10 18:43:05,181 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [6b471230] Running command: DisconnectStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:05,185 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-4) [6b471230] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 31394864 >2014-09-10 18:43:05,319 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-4) [6b471230] FINISH, DisconnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 31394864 >2014-09-10 18:43:05,617 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-11) [6eea3681] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:05,682 INFO [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand] (ajp-/127.0.0.1:8702-11) [1e37b1b4] Lock Acquired to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >, sharedLocks= ] >2014-09-10 18:43:05,753 INFO [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand] (ajp-/127.0.0.1:8702-11) [1e37b1b4] Running command: AddStoragePoolWithStoragesCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePoolAction group CREATE_STORAGE_POOL with role type ADMIN >2014-09-10 18:43:06,021 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (ajp-/127.0.0.1:8702-11) [3be86080] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:06,027 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-11) [3be86080] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3065d3f6 >2014-09-10 18:43:06,059 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-11) [3be86080] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 3065d3f6 >2014-09-10 18:43:06,066 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand] (ajp-/127.0.0.1:8702-11) [3be86080] START, CreateStoragePoolVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId=00000002-0002-0002-0002-0000000001ad, storagePoolName=Default, masterDomainId=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, domainsIdList=[eb633636-3019-47a9-9f5a-fc81f8dfd7dc], masterVersion=1), log id: 30bab046 >2014-09-10 18:43:27,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStoragePoolVDSCommand] (ajp-/127.0.0.1:8702-11) [3be86080] FINISH, CreateStoragePoolVDSCommand, log id: 30bab046 >2014-09-10 18:43:27,623 INFO [org.ovirt.engine.core.bll.storage.AddStoragePoolWithStoragesCommand] (ajp-/127.0.0.1:8702-11) [3be86080] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >, sharedLocks= ] >2014-09-10 18:43:27,627 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [6cbd2ead] Lock Acquired to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-10 18:43:27,634 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [6cbd2ead] Running command: ActivateStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:27,672 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [6cbd2ead] Lock freed to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-10 18:43:27,673 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [6cbd2ead] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/10/14 6:43 PM >2014-09-10 18:43:27,680 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-4) [75baf291] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-10 18:43:27,696 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-4) [75baf291] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 51ca2e55 >2014-09-10 18:43:27,710 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-4) [75baf291] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 51ca2e55 >2014-09-10 18:43:27,716 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-11) [6cbd2ead] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc), log id: 7c69201b >2014-09-10 18:43:27,812 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-10 18:43:27,840 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [788f6ac6] Correlation ID: 788f6ac6, Call Stack: null, Custom Event ID: -1, Message: Data Center is being initialized, please wait for initialization to complete. >2014-09-10 18:43:27,855 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (ajp-/127.0.0.1:8702-11) [788f6ac6] hostFromVds::selectedVds - rack, spmStatus Unknown_Pool, storage pool Default >2014-09-10 18:43:27,883 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vdsId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 709bc27d >2014-09-10 18:43:28,212 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] FINISH, ConnectStoragePoolVDSCommand, log id: 709bc27d >2014-09-10 18:43:28,217 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (ajp-/127.0.0.1:8702-11) [788f6ac6] hostFromVds::Connected host to pool - selectedVds - rack, spmStatus Free, storage pool Default >2014-09-10 18:43:28,235 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (ajp-/127.0.0.1:8702-11) [788f6ac6] starting spm on vds rack, storage pool Default, prevId -1, LVER -1 >2014-09-10 18:43:28,239 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] START, SpmStartVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=-1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 1a309793 >2014-09-10 18:43:28,248 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] spmStart polling started: taskId = f581af41-2b11-4336-a6c2-cbca753d9fb7 >2014-09-10 18:43:49,371 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] spmStart polling ended: taskId = f581af41-2b11-4336-a6c2-cbca753d9fb7 task status = finished >2014-09-10 18:43:49,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] spmStart polling ended, spm status: SPM >2014-09-10 18:43:49,386 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=f581af41-2b11-4336-a6c2-cbca753d9fb7), log id: 3e1df2f3 >2014-09-10 18:43:49,391 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] FINISH, HSMClearTaskVDSCommand, log id: 3e1df2f3 >2014-09-10 18:43:49,391 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@5b949c90, log id: 1a309793 >2014-09-10 18:43:49,398 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (ajp-/127.0.0.1:8702-11) [788f6ac6] Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 18:43:49,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [788f6ac6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-10 18:43:49,427 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-10 18:43:49,561 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] FINISH, ActivateStorageDomainVDSCommand, return: true, log id: 7c69201b >2014-09-10 18:43:49,561 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] ActivateStorage Domain. After Connect all hosts to pool. Time:9/10/14 6:43 PM >2014-09-10 18:43:49,600 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-16) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vdsId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 78897137 >2014-09-10 18:43:49,797 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-16) FINISH, ConnectStoragePoolVDSCommand, log id: 78897137 >2014-09-10 18:43:49,798 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-11) [788f6ac6] ActivateStorage Domain. After change storage pool status in vds. Time:9/10/14 6:43 PM >2014-09-10 18:43:49,828 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [788f6ac6] Correlation ID: 6cbd2ead, Call Stack: null, Custom Event ID: -1, Message: Storage Domain master (Data Center Default) was activated by admin >2014-09-10 18:43:49,869 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [788f6ac6] Correlation ID: 1e37b1b4, Job ID: eb718e6c-fad0-46c0-b514-b2f5b3af9bde, Call Stack: null, Custom Event ID: -1, Message: Storage Domains were attached to Data Center Default by admin >2014-09-10 18:43:49,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [788f6ac6] Correlation ID: 6eea3681, Job ID: eb718e6c-fad0-46c0-b514-b2f5b3af9bde, Call Stack: null, Custom Event ID: -1, Message: Storage Domain master was attached to Data Center Default by admin >2014-09-10 18:46:23,291 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (ajp-/127.0.0.1:8702-4) [4cdde22] Lock Acquired to object EngineLock [exclusiveLocks= key: e8715843-38c5-4188-b00e-b9ceadce2714 value: VM >, sharedLocks= ] >2014-09-10 18:46:23,417 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-6-thread-38) [4cdde22] Running command: RemoveVmCommand internal: false. Entities affected : ID: e8715843-38c5-4188-b00e-b9ceadce2714 Type: VMAction group DELETE_VM with role type USER >2014-09-10 18:46:23,421 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-38) [4cdde22] START, SetVmStatusVDSCommand( vmId = e8715843-38c5-4188-b00e-b9ceadce2714, status = ImageLocked, exit status = Normal), log id: 3c828193 >2014-09-10 18:46:23,460 INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (org.ovirt.thread.pool-6-thread-38) [4cdde22] FINISH, SetVmStatusVDSCommand, log id: 3c828193 >2014-09-10 18:46:23,465 INFO [org.ovirt.engine.core.bll.RemoveVmCommand] (org.ovirt.thread.pool-6-thread-38) [4cdde22] Lock freed to object EngineLock [exclusiveLocks= key: e8715843-38c5-4188-b00e-b9ceadce2714 value: VM >, sharedLocks= ] >2014-09-10 18:46:23,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-38) [4cdde22] Correlation ID: 4cdde22, Job ID: 35584d42-e9ee-4996-8697-d08ae4b5bc8d, Call Stack: null, Custom Event ID: -1, Message: VM external-341 was successfully removed. >2014-09-10 18:46:38,637 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [4fee3ba6] Lock Acquired to object EngineLock [exclusiveLocks= key: f20 value: VM_NAME >, sharedLocks= ] >2014-09-10 18:46:38,731 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [4fee3ba6] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-09-10 18:46:39,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [4fee3ba6] Correlation ID: 4fee3ba6, Job ID: 3b94b04f-45c6-4aa7-bcdc-078468c6ada7, Call Stack: null, Custom Event ID: -1, Message: VM f20 was created by admin. >2014-09-10 18:46:39,226 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-2) [4fee3ba6] Lock freed to object EngineLock [exclusiveLocks= key: f20 value: VM_NAME >, sharedLocks= ] >2014-09-10 18:46:41,321 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-3) [19aaf070] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: 0000000e-000e-000e-000e-0000000000e4 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-09-10 18:46:41,514 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-3) [7004db89] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VM >2014-09-10 18:46:41,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [7004db89] Correlation ID: 7004db89, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM f20. (User: admin) >2014-09-10 18:46:41,639 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [7004db89] Correlation ID: 19aaf070, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM f20. (User: admin) >2014-09-10 18:46:41,781 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-5) [58466dc6] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER >2014-09-10 18:46:46,366 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-1) [ff0240b] Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM_DISK_BOOT >, sharedLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >] >2014-09-10 18:46:46,429 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-1) [ff0240b] Running command: AddDiskCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group CREATE_DISK with role type USER >2014-09-10 18:46:46,607 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-1) [456556fb] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:46:46,702 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-1) [456556fb] Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM_DISK_BOOT >, sharedLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >] >2014-09-10 18:46:46,705 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-1) [456556fb] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 72c1d2e5-3906-472a-884a-beaf8178a948, imageSizeInBytes = 21474836480, volumeFormat = RAW, newImageId = 455469db-f6b6-4e6a-b906-8dc04ce3907a, newImageDescription = ), log id: 12717580 >2014-09-10 18:46:46,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-1) [456556fb] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-09-10 18:46:47,056 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-1) [456556fb] FINISH, CreateImageVDSCommand, return: 455469db-f6b6-4e6a-b906-8dc04ce3907a, log id: 12717580 >2014-09-10 18:46:47,320 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-1) [456556fb] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 675df457-0b1f-4609-ae17-224660ec9d2d >2014-09-10 18:46:47,320 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-1) [456556fb] CommandMultiAsyncTasks::AttachTask: Attaching task ac26e32d-bd7d-44bb-8969-3c61ed143807 to command 675df457-0b1f-4609-ae17-224660ec9d2d. >2014-09-10 18:46:47,364 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-1) [456556fb] Adding task ac26e32d-bd7d-44bb-8969-3c61ed143807 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-10 18:46:47,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [456556fb] Correlation ID: ff0240b, Job ID: 8bcde22a-71b9-415f-a150-62f9da7bbcc0, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of f20_Disk1 was initiated on VM f20 by admin. >2014-09-10 18:46:47,498 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-1) [456556fb] BaseAsyncTask::startPollingTask: Starting to poll task ac26e32d-bd7d-44bb-8969-3c61ed143807. >2014-09-10 18:46:52,219 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [70383b25] Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-10 18:46:52,281 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [70383b25] START, IsVmDuringInitiatingVDSCommand( vmId = 349f8284-643b-4d9e-b208-aff7af2b5fb3), log id: 5cb8b278 >2014-09-10 18:46:52,281 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-4) [70383b25] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5cb8b278 >2014-09-10 18:46:52,285 WARN [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [70383b25] CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,ACTION_TYPE_FAILED_DISKS_LOCKED,$diskAliases f20_Disk1 >2014-09-10 18:46:52,285 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-4) [70383b25] Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-10 18:46:53,677 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-94) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-09-10 18:46:53,686 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-94) SPMAsyncTask::PollTask: Polling task ac26e32d-bd7d-44bb-8969-3c61ed143807 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-10 18:46:53,698 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-94) BaseAsyncTask::onTaskEndSuccess: Task ac26e32d-bd7d-44bb-8969-3c61ed143807 (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-10 18:46:53,699 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-94) CommandAsyncTask::endActionIfNecessary: All tasks of command 675df457-0b1f-4609-ae17-224660ec9d2d has ended -> executing endAction >2014-09-10 18:46:53,700 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-94) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 675df457-0b1f-4609-ae17-224660ec9d2d): calling endAction . >2014-09-10 18:46:53,701 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-09-10 18:46:53,761 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-6-thread-3) [ff0240b] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-09-10 18:46:53,787 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-6-thread-3) [456556fb] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-09-10 18:46:53,837 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) [456556fb] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 72c1d2e5-3906-472a-884a-beaf8178a948, imageId = 455469db-f6b6-4e6a-b906-8dc04ce3907a), log id: 3b562cb4 >2014-09-10 18:46:53,912 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) [456556fb] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@5c3fb6c, log id: 3b562cb4 >2014-09-10 18:46:53,965 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-3) Correlation ID: ff0240b, Call Stack: null, Custom Event ID: -1, Message: The disk f20_Disk1 was successfully added to VM f20. >2014-09-10 18:46:53,966 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-09-10 18:46:53,966 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-09-10 18:46:53,972 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-3) SPMAsyncTask::ClearAsyncTask: Attempting to clear task ac26e32d-bd7d-44bb-8969-3c61ed143807 >2014-09-10 18:46:53,974 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = ac26e32d-bd7d-44bb-8969-3c61ed143807), log id: 2ec9cff8 >2014-09-10 18:46:53,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=ac26e32d-bd7d-44bb-8969-3c61ed143807), log id: 4374bac7 >2014-09-10 18:46:54,010 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, HSMClearTaskVDSCommand, log id: 4374bac7 >2014-09-10 18:46:54,010 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, SPMClearTaskVDSCommand, log id: 2ec9cff8 >2014-09-10 18:46:54,063 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-3) BaseAsyncTask::removeTaskFromDB: Removed task ac26e32d-bd7d-44bb-8969-3c61ed143807 from DataBase >2014-09-10 18:46:54,063 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-3) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 675df457-0b1f-4609-ae17-224660ec9d2d >2014-09-10 18:46:59,589 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-6) [7bb26db6] Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-10 18:46:59,635 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [7bb26db6] START, IsVmDuringInitiatingVDSCommand( vmId = 349f8284-643b-4d9e-b208-aff7af2b5fb3), log id: 8a745d1 >2014-09-10 18:46:59,637 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [7bb26db6] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 8a745d1 >2014-09-10 18:46:59,792 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] Running command: RunVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-10 18:46:59,916 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] START, UpdateVmDynamicDataVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@9d03fbf8), log id: 79e38781 >2014-09-10 18:46:59,925 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] FINISH, UpdateVmDynamicDataVDSCommand, log id: 79e38781 >2014-09-10 18:46:59,944 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] START, CreateVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 14cf0741 >2014-09-10 18:47:00,054 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] START, CreateVmFromCloudInitVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 4fe9cd85 >2014-09-10 18:47:00,227 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{shared=false, iface=ide, index=3, specParams={vmPayload={file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgInV1aWQiIDogIjg1MWRjYWM0LWQyNWMtNGUyZS1hZTM1LTI4NDdjMzY0MDExOSIsCiAgIm1l >dGEiIDogewogICAgImVzc2VudGlhbCIgOiAiZmFsc2UiLAogICAgInJvbGUiIDogInNlcnZlciIs >CiAgICAiZHNtb2RlIiA6ICJsb2NhbCIKICB9Cn0= >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >}, volId=config-2}}, path=, device=cdrom, type=disk, readonly=true, deviceId=f8f952f1-77fb-42df-a70c-68ef162f7c8e}, {specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=798508b7-1373-4465-a7b4-1c5f25d38bd6}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=fa22ef87-6e17-4c22-b6fe-50d5d4ae9758}, {shared=false, index=0, volumeID=455469db-f6b6-4e6a-b906-8dc04ce3907a, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=72c1d2e5-3906-472a-884a-beaf8178a948, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=72c1d2e5-3906-472a-884a-beaf8178a948}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:90, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92a2107d-cec5-47a5-96bf-3dd4ec81fb6c}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=3838190b-abba-4a39-b5b4-3845d56b684b}, {specParams={}, device=virtio-serial, type=controller, deviceId=1a591966-8fde-4c36-a832-8c75d400b1c1}],vmName=f20,cpuType=SandyBridge,fileTransferEnable=true >2014-09-10 18:47:00,295 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] FINISH, CreateVmFromCloudInitVDSCommand, log id: 4fe9cd85 >2014-09-10 18:47:00,322 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 14cf0741 >2014-09-10 18:47:00,322 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-10 18:47:00,373 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-5) [7bb26db6] Correlation ID: 7bb26db6, Job ID: b93d3eb6-3482-4bce-9cae-02ac281efc09, Call Stack: null, Custom Event ID: -1, Message: VM f20 was started by admin (Host: rack). >2014-09-10 18:47:00,974 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[349f8284-643b-4d9e-b208-aff7af2b5fb3]), log id: 6dc67ec1 >2014-09-10 18:47:00,985 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=0, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=WaitForLaunch, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@2abe935, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 6dc67ec1 >2014-09-10 18:47:00,990 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon} >2014-09-10 18:47:00,990 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a scsi Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {index=0, specParams={}, deviceType=controller, deviceId=3838190b-abba-4a39-b5b4-3845d56b684b, device=scsi, model=virtio-scsi, type=controller} >2014-09-10 18:47:00,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a virtio-serial Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=virtio-serial, specParams={}, deviceType=controller, type=controller, deviceId=1a591966-8fde-4c36-a832-8c75d400b1c1} >2014-09-10 18:47:00,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a qxl Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=qxl, specParams={vram=32768, heads=1}, deviceType=video, type=video, deviceId=798508b7-1373-4465-a7b4-1c5f25d38bd6} >2014-09-10 18:47:00,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, type=graphics} >2014-09-10 18:47:00,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a bridge Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {nicModel=pv, macAddr=00:1a:4a:d0:3f:90, linkActive=true, network=rhevm, filter=vdsm-no-mac-spoofing, specParams={inbound={}, outbound={}}, deviceType=interface, deviceId=92a2107d-cec5-47a5-96bf-3dd4ec81fb6c, device=bridge, type=interface} >2014-09-10 18:47:00,991 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a cdrom Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {index=3, iface=ide, specParams={vmPayload={volId=config-2, file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgInV1aWQiIDogIjg1MWRjYWM0LWQyNWMtNGUyZS1hZTM1LTI4NDdjMzY0MDExOSIsCiAgIm1l >dGEiIDogewogICAgImVzc2VudGlhbCIgOiAiZmFsc2UiLAogICAgInJvbGUiIDogInNlcnZlciIs >CiAgICAiZHNtb2RlIiA6ICJsb2NhbCIKICB9Cn0= >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >}}}, readonly=true, deviceType=disk, deviceId=f8f952f1-77fb-42df-a70c-68ef162f7c8e, path=/var/run/vdsm/payload/349f8284-643b-4d9e-b208-aff7af2b5fb3.6fc3ba5ac05fa168176a77d545b12a08.img, device=cdrom, shared=false, type=disk} >2014-09-10 18:47:00,992 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a cdrom Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {index=2, iface=ide, specParams={path=}, readonly=true, deviceType=disk, deviceId=fa22ef87-6e17-4c22-b6fe-50d5d4ae9758, path=, device=cdrom, shared=false, type=disk} >2014-09-10 18:47:00,992 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Received a disk Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {volumeInfo={path=/rhev/data-center/mnt/10.34.63.202:_mnt_export_nfs_lv3_pbenas_35/eb633636-3019-47a9-9f5a-fc81f8dfd7dc/images/72c1d2e5-3906-472a-884a-beaf8178a948/455469db-f6b6-4e6a-b906-8dc04ce3907a, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, volType=path, volumeID=455469db-f6b6-4e6a-b906-8dc04ce3907a, imageID=72c1d2e5-3906-472a-884a-beaf8178a948}, index=0, iface=virtio, apparentsize=21474836480, imageID=72c1d2e5-3906-472a-884a-beaf8178a948, readonly=false, deviceType=disk, shared=false, truesize=0, type=disk, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, reqsize=0, format=raw, deviceId=72c1d2e5-3906-472a-884a-beaf8178a948, poolID=00000002-0002-0002-0002-0000000001ad, device=disk, path=/rhev/data-center/00000002-0002-0002-0002-0000000001ad/eb633636-3019-47a9-9f5a-fc81f8dfd7dc/images/72c1d2e5-3906-472a-884a-beaf8178a948/455469db-f6b6-4e6a-b906-8dc04ce3907a, propagateErrors=off, optional=false, bootOrder=1, volumeID=455469db-f6b6-4e6a-b906-8dc04ce3907a, specParams={}, volumeChain=[Ljava.lang.Object;@64d56bb6} >2014-09-10 18:47:04,102 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from WaitForLaunch --> PoweringUp >2014-09-10 18:47:04,102 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[349f8284-643b-4d9e-b208-aff7af2b5fb3]), log id: 2dbb9700 >2014-09-10 18:47:04,122 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@d2773fe, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 2dbb9700 >2014-09-10 18:47:04,124 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-10 18:47:04,127 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-10 18:47:06,615 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-11) [1260f22c] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:06,631 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [1260f22c] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=uem/uhQb2B5q, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 6c21a29f >2014-09-10 18:47:06,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [1260f22c] FINISH, SetVmTicketVDSCommand, log id: 6c21a29f >2014-09-10 18:47:06,736 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [1260f22c] Correlation ID: 1260f22c, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:18,355 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-9) [4a7f0c5a] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:18,369 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [4a7f0c5a] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=olbya4ZkWNvh, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 2a0555af >2014-09-10 18:47:18,406 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [4a7f0c5a] FINISH, SetVmTicketVDSCommand, log id: 2a0555af >2014-09-10 18:47:18,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [4a7f0c5a] Correlation ID: 4a7f0c5a, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:20,023 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-4) [10d4ee8a] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:20,035 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [10d4ee8a] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=6xPShI9xGq8E, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 54d2dc94 >2014-09-10 18:47:20,073 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [10d4ee8a] FINISH, SetVmTicketVDSCommand, log id: 54d2dc94 >2014-09-10 18:47:20,093 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [10d4ee8a] Correlation ID: 10d4ee8a, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:20,126 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-11) [2186f002] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:20,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [2186f002] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=m03YFvDUUwVd, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5b5ae6d8 >2014-09-10 18:47:20,183 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [2186f002] FINISH, SetVmTicketVDSCommand, log id: 5b5ae6d8 >2014-09-10 18:47:20,224 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [2186f002] Correlation ID: 2186f002, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:20,288 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-7) [ae87c0d] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:20,325 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [ae87c0d] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=jap4dctewYzy, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 7d1811c0 >2014-09-10 18:47:20,357 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [ae87c0d] FINISH, SetVmTicketVDSCommand, log id: 7d1811c0 >2014-09-10 18:47:20,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [ae87c0d] Correlation ID: ae87c0d, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:21,660 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-2) [4830f60] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:21,672 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-2) [4830f60] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=h1DzXMVjpgDH, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 8621f3c >2014-09-10 18:47:21,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-2) [4830f60] FINISH, SetVmTicketVDSCommand, log id: 8621f3c >2014-09-10 18:47:21,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [4830f60] Correlation ID: 4830f60, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,181 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-21) [3da122b4] Start running CanDoAction for command number 1/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,188 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-21) [3da122b4] Finish handling CanDoAction for command number 1/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,189 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-20) [1ded7708] Start running CanDoAction for command number 2/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,189 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-22) [5121bd91] Start running CanDoAction for command number 3/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,190 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-23) [7e24a433] Start running CanDoAction for command number 4/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,190 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-24) [179850f8] Start running CanDoAction for command number 5/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,199 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-24) [179850f8] Finish handling CanDoAction for command number 5/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,204 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-22) [5121bd91] Finish handling CanDoAction for command number 3/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,206 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-20) [1ded7708] Finish handling CanDoAction for command number 2/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,208 INFO [org.ovirt.engine.core.bll.MultipleActionsRunner] (org.ovirt.thread.pool-6-thread-23) [7e24a433] Finish handling CanDoAction for command number 4/5 (Command type: SetVmTicket) >2014-09-10 18:47:22,211 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [3da122b4] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,224 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [3da122b4] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=AtC1f4EoEyFM, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5fe3d7ab >2014-09-10 18:47:22,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [3da122b4] FINISH, SetVmTicketVDSCommand, log id: 5fe3d7ab >2014-09-10 18:47:22,276 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [3da122b4] Correlation ID: 3da122b4, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,276 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [1ded7708] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,293 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [1ded7708] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=2FdGvQM3hZiE, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 5fff74a8 >2014-09-10 18:47:22,325 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [1ded7708] FINISH, SetVmTicketVDSCommand, log id: 5fff74a8 >2014-09-10 18:47:22,331 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-4) [4fbb2bce] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,351 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [1ded7708] Correlation ID: 1ded7708, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,351 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [5121bd91] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,368 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [4fbb2bce] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=3A+4MSn4OCNt, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 52676db9 >2014-09-10 18:47:22,408 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-4) [4fbb2bce] FINISH, SetVmTicketVDSCommand, log id: 52676db9 >2014-09-10 18:47:22,416 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [5121bd91] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=gGlby934+4ZC, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 2b038397 >2014-09-10 18:47:22,445 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4fbb2bce] Correlation ID: 4fbb2bce, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,450 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [5121bd91] FINISH, SetVmTicketVDSCommand, log id: 2b038397 >2014-09-10 18:47:22,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [5121bd91] Correlation ID: 5121bd91, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,491 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [7e24a433] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [7e24a433] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=uLAyRGlNmsVt, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 2cc9a626 >2014-09-10 18:47:22,551 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [7e24a433] FINISH, SetVmTicketVDSCommand, log id: 2cc9a626 >2014-09-10 18:47:22,568 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [7e24a433] Correlation ID: 7e24a433, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:22,569 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [179850f8] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:22,581 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [179850f8] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=lO10o32hcLFw, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 4462efa9 >2014-09-10 18:47:22,618 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [179850f8] FINISH, SetVmTicketVDSCommand, log id: 4462efa9 >2014-09-10 18:47:22,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [179850f8] Correlation ID: 179850f8, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:42,858 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-10) [5b21bf60] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:42,871 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-10) [5b21bf60] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=xe/vXBL6uMO4, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: ba13a47 >2014-09-10 18:47:42,919 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-10) [5b21bf60] FINISH, SetVmTicketVDSCommand, log id: ba13a47 >2014-09-10 18:47:42,936 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [5b21bf60] Correlation ID: 5b21bf60, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:43,450 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [462338ff] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:43,462 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [462338ff] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=8WQhlXN8/7aV, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 21e98b98 >2014-09-10 18:47:43,511 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [462338ff] FINISH, SetVmTicketVDSCommand, log id: 21e98b98 >2014-09-10 18:47:43,527 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [462338ff] Correlation ID: 462338ff, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:43,887 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-6) [3634c718] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:47:43,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-6) [3634c718] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=ZFVPNQGT0ng4, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 906b9ec >2014-09-10 18:47:43,944 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-6) [3634c718] FINISH, SetVmTicketVDSCommand, log id: 906b9ec >2014-09-10 18:47:43,961 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [3634c718] Correlation ID: 3634c718, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:47:45,372 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-10 18:47:45,372 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-10 18:48:02,058 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-43) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from PoweringUp --> Up >2014-09-10 18:48:02,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: 7bb26db6, Job ID: b93d3eb6-3482-4bce-9cae-02ac281efc09, Call Stack: null, Custom Event ID: -1, Message: VM f20 started on Host rack >2014-09-10 18:48:17,164 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-72) Setting new tasks map. The map contains now 0 tasks >2014-09-10 18:48:17,164 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-72) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-10 18:52:43,304 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-11) [5697003f] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-10 18:52:43,318 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [5697003f] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=do4ivSBy+uFU, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 4d8e32d1 >2014-09-10 18:52:43,370 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [5697003f] FINISH, SetVmTicketVDSCommand, log id: 4d8e32d1 >2014-09-10 18:52:43,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [5697003f] Correlation ID: 5697003f, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-10 18:52:49,110 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-10 18:52:49,111 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-10 18:59:14,726 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-62) Principal admin::internal synchronized >2014-09-10 18:59:15,208 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) Attempting to update VMs/Templates Ovf. >2014-09-10 18:59:15,584 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [6cac33b4] Lock Acquired to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 18:59:15,586 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [6cac33b4] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:15,599 INFO [org.ovirt.engine.core.bll.CreateOvfStoresForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [117db57f] Running command: CreateOvfStoresForStorageDomainCommand internal: true. >2014-09-10 18:59:15,608 INFO [org.ovirt.engine.core.bll.CreateOvfVolumeForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [3b4405bc] Running command: CreateOvfVolumeForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:15,680 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (DefaultQuartzScheduler_Worker-59) [2b61d806] Running command: AddDiskCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group CREATE_DISK with role type USER >2014-09-10 18:59:15,720 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:15,741 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageSizeInBytes = 134217728, volumeFormat = RAW, newImageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0, newImageDescription = OVF_STORE), log id: 55ce5f3 >2014-09-10 18:59:15,745 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-09-10 18:59:16,619 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] FINISH, CreateImageVDSCommand, return: 285be822-e08e-424e-ac3a-f6cdfbd6d0f0, log id: 55ce5f3 >2014-09-10 18:59:16,636 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 8f5da951-e9dc-48e9-a9d2-a19c41c64002 >2014-09-10 18:59:16,637 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] CommandMultiAsyncTasks::AttachTask: Attaching task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 to command 8f5da951-e9dc-48e9-a9d2-a19c41c64002. >2014-09-10 18:59:16,659 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] Adding task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-10 18:59:16,694 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [1b8eb09d] Correlation ID: 2b61d806, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of 'OVF_STORE' was initiated by <UNKNOWN>. >2014-09-10 18:59:16,716 INFO [org.ovirt.engine.core.bll.CreateOvfVolumeForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [625ca0b9] Running command: CreateOvfVolumeForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:16,771 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (DefaultQuartzScheduler_Worker-59) [5d8bd703] Running command: AddDiskCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group CREATE_DISK with role type USER >2014-09-10 18:59:16,841 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:16,863 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageSizeInBytes = 134217728, volumeFormat = RAW, newImageId = 32717d52-27ce-439e-8803-74ee646b94ea, newImageDescription = OVF_STORE), log id: 26127a27 >2014-09-10 18:59:16,866 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-09-10 18:59:17,781 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] FINISH, CreateImageVDSCommand, return: 32717d52-27ce-439e-8803-74ee646b94ea, log id: 26127a27 >2014-09-10 18:59:17,788 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] CommandMultiAsyncTasks::AttachTask: Attaching task 2bef6829-ef47-4036-94da-eb52696a1c4e to command 8f5da951-e9dc-48e9-a9d2-a19c41c64002. >2014-09-10 18:59:17,816 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] Adding task 2bef6829-ef47-4036-94da-eb52696a1c4e (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-10 18:59:17,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] Correlation ID: 5d8bd703, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of 'OVF_STORE' was initiated by <UNKNOWN>. >2014-09-10 18:59:17,873 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] BaseAsyncTask::startPollingTask: Starting to poll task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8. >2014-09-10 18:59:17,873 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] BaseAsyncTask::startPollingTask: Starting to poll task 2bef6829-ef47-4036-94da-eb52696a1c4e. >2014-09-10 18:59:17,875 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-59) [71ddfe6c] Lock freed to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 18:59:23,754 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-09-10 18:59:23,759 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-85) SPMAsyncTask::PollTask: Polling task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-09-10 18:59:23,759 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-85) SPMAsyncTask::PollTask: Polling task 2bef6829-ef47-4036-94da-eb52696a1c4e (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-09-10 18:59:23,760 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-85) Finished polling Tasks, will poll again in 10 seconds. >2014-09-10 18:59:53,778 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] SPMAsyncTask::PollTask: Polling task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-10 18:59:53,779 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] BaseAsyncTask::onTaskEndSuccess: Task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-10 18:59:53,779 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-64) [29539bf3] Task with DB Task ID 70d1663a-908f-4f91-a3ff-01ddc71f3a62 and VDSM Task ID 2bef6829-ef47-4036-94da-eb52696a1c4e is in state Polling. End action for command 8f5da951-e9dc-48e9-a9d2-a19c41c64002 will proceed when all the entitys tasks are completed. >2014-09-10 18:59:53,779 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] SPMAsyncTask::PollTask: Polling task 2bef6829-ef47-4036-94da-eb52696a1c4e (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-10 18:59:53,780 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] BaseAsyncTask::onTaskEndSuccess: Task 2bef6829-ef47-4036-94da-eb52696a1c4e (Parent Command CreateOvfStoresForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-10 18:59:53,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] CommandAsyncTask::endActionIfNecessary: All tasks of command 8f5da951-e9dc-48e9-a9d2-a19c41c64002 has ended -> executing endAction >2014-09-10 18:59:53,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-64) [29539bf3] CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 8f5da951-e9dc-48e9-a9d2-a19c41c64002): calling endAction . >2014-09-10 18:59:53,780 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-24) [29539bf3] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction CreateOvfStoresForStorageDomain, executionIndex: 0 >2014-09-10 18:59:53,782 INFO [org.ovirt.engine.core.bll.CreateOvfStoresForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] Ending command successfully: org.ovirt.engine.core.bll.CreateOvfStoresForStorageDomainCommand >2014-09-10 18:59:53,803 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-6-thread-24) [71ddfe6c] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-09-10 18:59:53,818 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-24) [71ddfe6c] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 5def0657 >2014-09-10 18:59:53,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-24) [71ddfe6c] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@c875a16c, log id: 5def0657 >2014-09-10 18:59:53,920 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-6-thread-24) [1b8eb09d] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-09-10 18:59:53,935 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-24) [1b8eb09d] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 7df475ed >2014-09-10 18:59:54,017 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-24) [1b8eb09d] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@6fdcf962, log id: 7df475ed >2014-09-10 18:59:54,121 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:54,198 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 6b3c1ceb >2014-09-10 18:59:54,198 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 18:59:54,198 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 18:59:54,380 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [117db57f] FINISH, SetVolumeDescriptionVDSCommand, log id: 6b3c1ceb >2014-09-10 18:59:54,467 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS_EXECUTION >] >2014-09-10 18:59:54,483 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:54,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 112784ce >2014-09-10 18:59:54,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] -- executeVdsBrokerCommand: >2014-09-10 18:59:54,497 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 18:59:54,500 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 18:59:54,500 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 18:59:54,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] FINISH, UploadStreamVDSCommand, log id: 112784ce >2014-09-10 18:59:54,501 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 18:59:54,527 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS_EXECUTION >] >2014-09-10 18:59:54,545 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-24) [76317ae] Removed task e8f717ac-bf6f-4ff9-8a55-f0514cb81a4c from DataBase >2014-09-10 18:59:54,566 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 6b8a0396 >2014-09-10 18:59:54,566 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 18:59:54,567 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 18:59:54,844 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (org.ovirt.thread.pool-6-thread-24) [76317ae] FINISH, SetVolumeDescriptionVDSCommand, log id: 6b8a0396 >2014-09-10 18:59:54,889 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS_EXECUTION >] >2014-09-10 18:59:54,898 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 18:59:54,900 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 3ccfe125 >2014-09-10 18:59:54,900 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] -- executeVdsBrokerCommand: >2014-09-10 18:59:54,900 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 18:59:54,901 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 18:59:54,901 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 18:59:54,901 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] FINISH, UploadStreamVDSCommand, log id: 3ccfe125 >2014-09-10 18:59:54,901 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 18:59:54,909 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS_EXECUTION >] >2014-09-10 18:59:54,915 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-24) [656d5e21] Removed task a623b3bc-1a3a-44f0-84cf-083515ecd8c0 from DataBase >2014-09-10 18:59:54,917 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateOvfStoresForStorageDomain completed, handling the result. >2014-09-10 18:59:54,917 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type CreateOvfStoresForStorageDomain succeeded, clearing tasks. >2014-09-10 18:59:54,917 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 2bef6829-ef47-4036-94da-eb52696a1c4e >2014-09-10 18:59:54,918 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 2bef6829-ef47-4036-94da-eb52696a1c4e), log id: a965f8e >2014-09-10 18:59:54,919 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=2bef6829-ef47-4036-94da-eb52696a1c4e), log id: 165b6910 >2014-09-10 18:59:55,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] FINISH, HSMClearTaskVDSCommand, log id: 165b6910 >2014-09-10 18:59:55,101 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] FINISH, SPMClearTaskVDSCommand, log id: a965f8e >2014-09-10 18:59:55,108 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] BaseAsyncTask::removeTaskFromDB: Removed task 2bef6829-ef47-4036-94da-eb52696a1c4e from DataBase >2014-09-10 18:59:55,108 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 >2014-09-10 18:59:55,108 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8), log id: 4c48969 >2014-09-10 18:59:55,110 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8), log id: 3194ef1 >2014-09-10 18:59:55,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] FINISH, HSMClearTaskVDSCommand, log id: 3194ef1 >2014-09-10 18:59:55,294 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-24) [656d5e21] FINISH, SPMClearTaskVDSCommand, log id: 4c48969 >2014-09-10 18:59:55,301 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] BaseAsyncTask::removeTaskFromDB: Removed task 89d706c7-dc1f-4fd5-a2da-0c5c6539b7f8 from DataBase >2014-09-10 18:59:55,301 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-24) [656d5e21] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 8f5da951-e9dc-48e9-a9d2-a19c41c64002 >2014-09-10 19:01:17,188 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-49) Setting new tasks map. The map contains now 0 tasks >2014-09-10 19:01:17,188 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-49) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-10 19:25:34,252 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-10 19:25:34,252 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-10 19:26:47,200 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-10 19:26:47,210 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 19:26:47,699 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-4) [30a0c212] Running command: LogoutBySessionCommand internal: false. >2014-09-10 19:26:47,702 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [4e00a95] Running command: LogoutUserCommand internal: false. >2014-09-10 19:26:47,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4e00a95] Correlation ID: 4e00a95, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 19:26:52,900 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-09-10 19:26:52,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-10 19:26:53,448 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-11) [1ad3791b] Running command: LogoutBySessionCommand internal: false. >2014-09-10 19:26:53,453 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [259c9deb] Running command: LogoutUserCommand internal: false. >2014-09-10 19:26:53,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [259c9deb] Correlation ID: 259c9deb, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 19:26:53,621 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-09-10 19:26:53,648 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-1) [4c78b302] Running command: LogoutBySessionCommand internal: false. >2014-09-10 19:26:53,651 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [7c52022] Running command: LogoutUserCommand internal: false. >2014-09-10 19:26:53,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [7c52022] Correlation ID: 7c52022, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-10 19:59:14,737 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-96) [21e034cd] Principal admin::internal synchronized >2014-09-10 19:59:17,876 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-09-10 19:59:17,976 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 19:59:17,979 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 19:59:18,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 75b07d80 >2014-09-10 19:59:18,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 19:59:18,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 19:59:18,262 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [2ba6a1fe] FINISH, SetVolumeDescriptionVDSCommand, log id: 75b07d80 >2014-09-10 19:59:18,320 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 19:59:18,323 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 2bf60050 >2014-09-10 19:59:18,324 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] -- executeVdsBrokerCommand: >2014-09-10 19:59:18,324 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 19:59:18,324 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 19:59:18,324 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 19:59:18,325 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] FINISH, UploadStreamVDSCommand, log id: 2bf60050 >2014-09-10 19:59:18,325 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 19:59:18,335 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 19:59:18,359 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-19) [ceb3043] Removed task e7fba5f2-d90f-40e6-8ab3-f08365f07012 from DataBase >2014-09-10 19:59:18,370 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 7414617 >2014-09-10 19:59:18,370 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 19:59:18,370 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 19:59:18,575 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-19) [ceb3043] FINISH, SetVolumeDescriptionVDSCommand, log id: 7414617 >2014-09-10 19:59:18,612 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 19:59:18,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 3275e4ab >2014-09-10 19:59:18,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] -- executeVdsBrokerCommand: >2014-09-10 19:59:18,615 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 19:59:18,616 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 19:59:18,616 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 19:59:18,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] FINISH, UploadStreamVDSCommand, log id: 3275e4ab >2014-09-10 19:59:18,616 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 19:59:18,631 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 19:59:18,632 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 19:59:18,632 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 19:59:18,632 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 19:59:18,633 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 19:59:18,651 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Removed task 97e8f6b4-8444-4539-af4b-b73fc5aebbcd from DataBase >2014-09-10 19:59:18,653 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 19:59:18,653 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 19:59:18,653 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 19:59:18,654 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 19:59:18,654 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-19) [3cbe4556] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 20:59:14,749 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-37) Principal admin::internal synchronized >2014-09-10 20:59:18,655 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) Attempting to update VMs/Templates Ovf. >2014-09-10 20:59:18,770 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 20:59:18,773 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 20:59:18,806 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 4536f987 >2014-09-10 20:59:18,806 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 20:59:18,806 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 20:59:18,885 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [20f48730] FINISH, SetVolumeDescriptionVDSCommand, log id: 4536f987 >2014-09-10 20:59:18,930 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 20:59:18,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 3c9f3b72 >2014-09-10 20:59:18,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] -- executeVdsBrokerCommand: >2014-09-10 20:59:18,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 20:59:18,933 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 20:59:18,933 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 20:59:18,933 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] FINISH, UploadStreamVDSCommand, log id: 3c9f3b72 >2014-09-10 20:59:18,934 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 20:59:18,943 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 20:59:18,974 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-52) [71b000fd] Removed task 6fa8ec3f-d062-4c09-b891-b2b4e5cb9e60 from DataBase >2014-09-10 20:59:18,992 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 626ac868 >2014-09-10 20:59:18,992 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 20:59:18,992 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 20:59:19,233 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [71b000fd] FINISH, SetVolumeDescriptionVDSCommand, log id: 626ac868 >2014-09-10 20:59:19,283 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 20:59:19,285 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 68d59b19 >2014-09-10 20:59:19,285 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] -- executeVdsBrokerCommand: >2014-09-10 20:59:19,285 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 20:59:19,286 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 20:59:19,286 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 20:59:19,286 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] FINISH, UploadStreamVDSCommand, log id: 68d59b19 >2014-09-10 20:59:19,287 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 20:59:19,297 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 20:59:19,297 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 20:59:19,297 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 20:59:19,297 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 20:59:19,297 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 20:59:19,305 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Removed task fe20e1e7-a96f-4789-becb-e1219088ecd0 from DataBase >2014-09-10 20:59:19,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 20:59:19,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 20:59:19,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 20:59:19,307 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 20:59:19,308 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-52) [23ec0f25] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 21:59:14,761 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-69) Principal admin::internal synchronized >2014-09-10 21:59:19,309 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-09-10 21:59:19,356 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 21:59:19,359 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 21:59:19,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 5c50b6fd >2014-09-10 21:59:19,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 21:59:19,389 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 21:59:19,638 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [171ce287] FINISH, SetVolumeDescriptionVDSCommand, log id: 5c50b6fd >2014-09-10 21:59:19,681 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 21:59:19,684 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 6676b00d >2014-09-10 21:59:19,684 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] -- executeVdsBrokerCommand: >2014-09-10 21:59:19,684 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 21:59:19,684 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 21:59:19,684 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 21:59:19,684 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] FINISH, UploadStreamVDSCommand, log id: 6676b00d >2014-09-10 21:59:19,685 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 21:59:19,693 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 21:59:19,700 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [2ee38638] Removed task c2424662-f6d1-4e84-9e34-6d0c518e8781 from DataBase >2014-09-10 21:59:19,705 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 7a9a651e >2014-09-10 21:59:19,705 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 21:59:19,705 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 21:59:19,971 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-95) [2ee38638] FINISH, SetVolumeDescriptionVDSCommand, log id: 7a9a651e >2014-09-10 21:59:20,018 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 21:59:20,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 4fecf321 >2014-09-10 21:59:20,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] -- executeVdsBrokerCommand: >2014-09-10 21:59:20,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 21:59:20,021 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 21:59:20,021 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 21:59:20,021 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] FINISH, UploadStreamVDSCommand, log id: 4fecf321 >2014-09-10 21:59:20,021 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 21:59:20,035 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 21:59:20,035 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 21:59:20,035 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 21:59:20,036 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 21:59:20,036 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 21:59:20,045 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Removed task c70bac20-a0ab-4dc2-a0e4-9ebbcf5ac23c from DataBase >2014-09-10 21:59:20,047 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 21:59:20,047 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 21:59:20,047 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 21:59:20,047 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-95) [54edecc9] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 21:59:20,048 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-95) [54edecc9] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 22:59:14,774 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-22) Principal admin::internal synchronized >2014-09-10 22:59:20,050 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-09-10 22:59:20,140 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 22:59:20,143 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 22:59:20,162 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 3bac9ac9 >2014-09-10 22:59:20,162 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 22:59:20,163 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 22:59:20,431 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [6d0d69] FINISH, SetVolumeDescriptionVDSCommand, log id: 3bac9ac9 >2014-09-10 22:59:20,473 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 22:59:20,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 9cfe71c >2014-09-10 22:59:20,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] -- executeVdsBrokerCommand: >2014-09-10 22:59:20,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 22:59:20,476 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 22:59:20,476 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 22:59:20,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] FINISH, UploadStreamVDSCommand, log id: 9cfe71c >2014-09-10 22:59:20,476 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 22:59:20,484 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 22:59:20,490 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-31) [5639c62f] Removed task 1fc7060f-ba50-47e3-b6ba-c5ca3b3e7c09 from DataBase >2014-09-10 22:59:20,494 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 24917ba6 >2014-09-10 22:59:20,494 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 22:59:20,495 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 22:59:20,689 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-31) [5639c62f] FINISH, SetVolumeDescriptionVDSCommand, log id: 24917ba6 >2014-09-10 22:59:20,727 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 22:59:20,729 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 39828a7e >2014-09-10 22:59:20,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] -- executeVdsBrokerCommand: >2014-09-10 22:59:20,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 22:59:20,730 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 22:59:20,730 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 22:59:20,731 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] FINISH, UploadStreamVDSCommand, log id: 39828a7e >2014-09-10 22:59:20,731 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 22:59:20,740 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 22:59:20,740 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 22:59:20,741 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 22:59:20,741 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 22:59:20,741 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 22:59:20,749 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Removed task ac6dbdc8-b3ef-4401-bda4-9fd0e4d2edf2 from DataBase >2014-09-10 22:59:20,750 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 22:59:20,751 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 22:59:20,751 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 22:59:20,751 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 22:59:20,751 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 23:59:14,785 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-58) Principal admin::internal synchronized >2014-09-10 23:59:20,752 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2014-09-10 23:59:20,797 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 23:59:20,800 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 23:59:20,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 414200ec >2014-09-10 23:59:20,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 23:59:20,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 23:59:21,093 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [6f29f31c] FINISH, SetVolumeDescriptionVDSCommand, log id: 414200ec >2014-09-10 23:59:21,144 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 23:59:21,146 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 68a9ac48 >2014-09-10 23:59:21,146 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] -- executeVdsBrokerCommand: >2014-09-10 23:59:21,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-10 23:59:21,147 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 23:59:21,147 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 23:59:21,147 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] FINISH, UploadStreamVDSCommand, log id: 68a9ac48 >2014-09-10 23:59:21,147 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 23:59:21,157 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 23:59:21,164 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-47) [3be19c92] Removed task 4a954ebb-e702-4e88-8ae4-0abc9bb159d2 from DataBase >2014-09-10 23:59:21,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 5455b2a0 >2014-09-10 23:59:21,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-10 23:59:21,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-10 23:59:21,374 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-47) [3be19c92] FINISH, SetVolumeDescriptionVDSCommand, log id: 5455b2a0 >2014-09-10 23:59:21,415 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-10 23:59:21,417 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 4bb5c763 >2014-09-10 23:59:21,417 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] -- executeVdsBrokerCommand: >2014-09-10 23:59:21,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-10 23:59:21,418 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-10 23:59:21,418 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-10 23:59:21,418 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] FINISH, UploadStreamVDSCommand, log id: 4bb5c763 >2014-09-10 23:59:21,418 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-10 23:59:21,426 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 23:59:21,426 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 23:59:21,426 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 23:59:21,427 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 23:59:21,427 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-10 23:59:21,437 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Removed task d43c2fbb-e66f-43a6-b743-1d53ea1dbe15 from DataBase >2014-09-10 23:59:21,438 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-10 23:59:21,438 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-10 23:59:21,439 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-10 23:59:21,439 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-47) [72af56aa] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-10 23:59:21,439 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-47) [72af56aa] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 00:59:14,799 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-11 00:59:21,440 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-11 00:59:21,478 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 00:59:21,480 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 00:59:21,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 2fc48d8 >2014-09-11 00:59:21,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 00:59:21,496 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 00:59:21,571 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [1aff5c60] FINISH, SetVolumeDescriptionVDSCommand, log id: 2fc48d8 >2014-09-11 00:59:21,609 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 00:59:21,611 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 36a593d7 >2014-09-11 00:59:21,612 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] -- executeVdsBrokerCommand: >2014-09-11 00:59:21,612 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 00:59:21,612 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 00:59:21,612 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 00:59:21,613 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] FINISH, UploadStreamVDSCommand, log id: 36a593d7 >2014-09-11 00:59:21,613 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 00:59:21,622 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 00:59:21,629 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-22) [101eb6d7] Removed task b5e17e8a-a805-46ec-be37-36399256e7b5 from DataBase >2014-09-11 00:59:21,633 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 78820a0a >2014-09-11 00:59:21,633 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 00:59:21,634 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 00:59:21,833 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-22) [101eb6d7] FINISH, SetVolumeDescriptionVDSCommand, log id: 78820a0a >2014-09-11 00:59:21,892 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 00:59:21,894 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 18305b06 >2014-09-11 00:59:21,894 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] -- executeVdsBrokerCommand: >2014-09-11 00:59:21,895 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 00:59:21,895 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 00:59:21,895 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 00:59:21,895 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] FINISH, UploadStreamVDSCommand, log id: 18305b06 >2014-09-11 00:59:21,895 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 00:59:21,903 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 00:59:21,903 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 00:59:21,903 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 00:59:21,904 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 00:59:21,904 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 00:59:21,919 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Removed task c6ce786e-1eea-4ba1-a934-718132505b82 from DataBase >2014-09-11 00:59:21,921 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 00:59:21,921 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 00:59:21,921 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 00:59:21,921 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 00:59:21,921 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-22) [4c2150b9] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 01:59:14,810 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-39) Principal admin::internal synchronized >2014-09-11 01:59:21,922 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VMs/Templates Ovf. >2014-09-11 01:59:22,126 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 01:59:22,129 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 01:59:22,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 4abde944 >2014-09-11 01:59:22,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 01:59:22,174 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 01:59:22,428 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [50b0ef7f] FINISH, SetVolumeDescriptionVDSCommand, log id: 4abde944 >2014-09-11 01:59:22,473 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 01:59:22,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 65b8b70 >2014-09-11 01:59:22,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] -- executeVdsBrokerCommand: >2014-09-11 01:59:22,476 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 01:59:22,477 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 01:59:22,477 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 01:59:22,477 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] FINISH, UploadStreamVDSCommand, log id: 65b8b70 >2014-09-11 01:59:22,477 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 01:59:22,487 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 01:59:22,494 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) [1178af23] Removed task 395c8ddf-453c-4bd3-8041-34c74cba0a81 from DataBase >2014-09-11 01:59:22,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 5da4fb26 >2014-09-11 01:59:22,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 01:59:22,500 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 01:59:22,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-54) [1178af23] FINISH, SetVolumeDescriptionVDSCommand, log id: 5da4fb26 >2014-09-11 01:59:22,769 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 01:59:22,772 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 11b2cd05 >2014-09-11 01:59:22,772 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] -- executeVdsBrokerCommand: >2014-09-11 01:59:22,772 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 01:59:22,773 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 01:59:22,773 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 01:59:22,773 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] FINISH, UploadStreamVDSCommand, log id: 11b2cd05 >2014-09-11 01:59:22,773 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 01:59:22,782 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 01:59:22,783 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 01:59:22,783 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 01:59:22,783 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 01:59:22,783 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 01:59:22,789 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Removed task 21b83f13-6b6d-474d-bb21-518e5c7179ce from DataBase >2014-09-11 01:59:22,791 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 01:59:22,791 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 01:59:22,791 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 01:59:22,791 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-54) [34f87778] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 01:59:22,791 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-54) [34f87778] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 02:59:14,822 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-11 02:59:22,792 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-11 02:59:22,829 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 02:59:22,832 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 02:59:22,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 3c864e9b >2014-09-11 02:59:22,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 02:59:22,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 02:59:22,927 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [5b73f716] FINISH, SetVolumeDescriptionVDSCommand, log id: 3c864e9b >2014-09-11 02:59:22,973 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 02:59:22,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 21c561c9 >2014-09-11 02:59:22,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] -- executeVdsBrokerCommand: >2014-09-11 02:59:22,976 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 02:59:22,977 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 02:59:22,977 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 02:59:22,977 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] FINISH, UploadStreamVDSCommand, log id: 21c561c9 >2014-09-11 02:59:22,977 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 02:59:22,987 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 02:59:22,994 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) [3ecc3941] Removed task 798f73e1-5041-4dfb-832f-acec1a552e44 from DataBase >2014-09-11 02:59:22,999 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 5e9ce3e9 >2014-09-11 02:59:22,999 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 02:59:23,000 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 02:59:23,096 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-76) [3ecc3941] FINISH, SetVolumeDescriptionVDSCommand, log id: 5e9ce3e9 >2014-09-11 02:59:23,131 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 02:59:23,134 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 2bfd42b9 >2014-09-11 02:59:23,134 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] -- executeVdsBrokerCommand: >2014-09-11 02:59:23,134 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 02:59:23,134 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 02:59:23,135 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 02:59:23,135 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] FINISH, UploadStreamVDSCommand, log id: 2bfd42b9 >2014-09-11 02:59:23,135 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 02:59:23,150 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 02:59:23,151 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 02:59:23,151 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 02:59:23,151 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 02:59:23,151 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 02:59:23,163 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Removed task 43efc9b5-2f9a-4889-936a-fb7bb8447133 from DataBase >2014-09-11 02:59:23,165 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 02:59:23,165 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 02:59:23,165 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 02:59:23,165 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 02:59:23,166 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-76) [48cc00fd] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-81) Start deleteAgedOutAuditLogs >2014-09-11 03:35:35,010 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-97) Start deleteAgedOutCommandEntities >2014-09-11 03:35:35,020 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-81) Finished deleteAgedOutAuditLogs >2014-09-11 03:35:35,021 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-97) Finished deleteAgedOutCommandEntities >2014-09-11 03:59:14,836 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) Principal admin::internal synchronized >2014-09-11 03:59:23,167 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-09-11 03:59:23,206 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 03:59:23,209 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 03:59:23,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 31cb6cae >2014-09-11 03:59:23,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 03:59:23,222 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 03:59:23,313 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [30895c33] FINISH, SetVolumeDescriptionVDSCommand, log id: 31cb6cae >2014-09-11 03:59:23,348 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 03:59:23,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 72446b7e >2014-09-11 03:59:23,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] -- executeVdsBrokerCommand: >2014-09-11 03:59:23,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 03:59:23,351 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 03:59:23,354 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 03:59:23,354 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] FINISH, UploadStreamVDSCommand, log id: 72446b7e >2014-09-11 03:59:23,354 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 03:59:23,362 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 03:59:23,368 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] Removed task 00c48ac6-be44-46bb-9e87-23677a6eb5d4 from DataBase >2014-09-11 03:59:23,377 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 3a6f5e00 >2014-09-11 03:59:23,377 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 03:59:23,377 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 03:59:23,470 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-26) [5c1bb97c] FINISH, SetVolumeDescriptionVDSCommand, log id: 3a6f5e00 >2014-09-11 03:59:23,512 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 03:59:23,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 5f4cac0 >2014-09-11 03:59:23,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] -- executeVdsBrokerCommand: >2014-09-11 03:59:23,514 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 03:59:23,515 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 03:59:23,515 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 03:59:23,515 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] FINISH, UploadStreamVDSCommand, log id: 5f4cac0 >2014-09-11 03:59:23,515 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 03:59:23,524 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 03:59:23,524 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 03:59:23,524 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 03:59:23,524 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 03:59:23,524 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 03:59:23,532 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Removed task 6d9e4501-3eb4-4810-a545-d7b4ec3f6706 from DataBase >2014-09-11 03:59:23,534 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 03:59:23,534 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 03:59:23,534 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 03:59:23,534 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 03:59:23,534 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-26) [2278a6f6] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 04:59:14,851 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) [5d4cd059] Principal admin::internal synchronized >2014-09-11 04:59:23,536 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-11 04:59:23,569 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 04:59:23,572 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 04:59:23,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 141ace23 >2014-09-11 04:59:23,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 04:59:23,592 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 04:59:23,788 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [31db181e] FINISH, SetVolumeDescriptionVDSCommand, log id: 141ace23 >2014-09-11 04:59:23,827 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 04:59:23,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: cdc1eed >2014-09-11 04:59:23,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] -- executeVdsBrokerCommand: >2014-09-11 04:59:23,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 04:59:23,830 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 04:59:23,830 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 04:59:23,830 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] FINISH, UploadStreamVDSCommand, log id: cdc1eed >2014-09-11 04:59:23,831 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 04:59:23,840 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 04:59:23,846 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-57) [2e9678ad] Removed task 419809aa-e6b0-455d-9121-23380a404a12 from DataBase >2014-09-11 04:59:23,872 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 6e6ccd0d >2014-09-11 04:59:23,872 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 04:59:23,873 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 04:59:24,090 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-57) [2e9678ad] FINISH, SetVolumeDescriptionVDSCommand, log id: 6e6ccd0d >2014-09-11 04:59:24,124 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 04:59:24,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 5ef5fe27 >2014-09-11 04:59:24,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] -- executeVdsBrokerCommand: >2014-09-11 04:59:24,126 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 04:59:24,127 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 04:59:24,127 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 04:59:24,127 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] FINISH, UploadStreamVDSCommand, log id: 5ef5fe27 >2014-09-11 04:59:24,128 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 04:59:24,137 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 04:59:24,137 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 04:59:24,137 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 04:59:24,138 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 04:59:24,138 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 04:59:24,144 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Removed task 8b2bbf54-4c5d-4dbb-8f1c-8fc41b23482c from DataBase >2014-09-11 04:59:24,146 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 04:59:24,146 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 04:59:24,146 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 04:59:24,146 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 04:59:24,147 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-57) [59fd1e0f] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 05:59:14,866 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-11 05:59:24,147 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) [21e034cd] Attempting to update VMs/Templates Ovf. >2014-09-11 05:59:24,187 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 05:59:24,191 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 05:59:24,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 1c151ca0 >2014-09-11 05:59:24,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 05:59:24,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 05:59:24,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [1bc203fe] FINISH, SetVolumeDescriptionVDSCommand, log id: 1c151ca0 >2014-09-11 05:59:24,418 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 05:59:24,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 77a84b7f >2014-09-11 05:59:24,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] -- executeVdsBrokerCommand: >2014-09-11 05:59:24,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 05:59:24,422 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 05:59:24,422 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 05:59:24,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] FINISH, UploadStreamVDSCommand, log id: 77a84b7f >2014-09-11 05:59:24,422 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 05:59:24,431 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 05:59:24,443 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) [38231c81] Removed task 6008dd4a-95dc-4b20-8324-203aa6c4d165 from DataBase >2014-09-11 05:59:24,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 4384d62f >2014-09-11 05:59:24,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 05:59:24,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 05:59:24,635 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-96) [38231c81] FINISH, SetVolumeDescriptionVDSCommand, log id: 4384d62f >2014-09-11 05:59:24,666 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 05:59:24,668 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 5d3cdc3 >2014-09-11 05:59:24,668 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] -- executeVdsBrokerCommand: >2014-09-11 05:59:24,668 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 05:59:24,669 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 05:59:24,669 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 05:59:24,669 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] FINISH, UploadStreamVDSCommand, log id: 5d3cdc3 >2014-09-11 05:59:24,669 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 05:59:24,677 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 05:59:24,677 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 05:59:24,678 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 05:59:24,678 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 05:59:24,678 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 05:59:24,685 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Removed task 9b34231f-fa90-4545-8647-ac44c26b07a8 from DataBase >2014-09-11 05:59:24,687 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 05:59:24,687 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 05:59:24,687 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 05:59:24,687 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-96) [39353a76] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 05:59:24,688 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-96) [39353a76] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 06:59:14,876 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-2) Principal admin::internal synchronized >2014-09-11 06:59:24,689 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-09-11 06:59:24,728 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 06:59:24,732 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 06:59:24,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: e0d1a4b >2014-09-11 06:59:24,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 06:59:24,749 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 06:59:24,847 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [1edb5ad4] FINISH, SetVolumeDescriptionVDSCommand, log id: e0d1a4b >2014-09-11 06:59:24,886 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 06:59:24,888 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 8d17683 >2014-09-11 06:59:24,888 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] -- executeVdsBrokerCommand: >2014-09-11 06:59:24,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 06:59:24,889 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 06:59:24,889 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 06:59:24,889 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] FINISH, UploadStreamVDSCommand, log id: 8d17683 >2014-09-11 06:59:24,889 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 06:59:24,899 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 06:59:24,909 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) [65c402b4] Removed task df74c07f-5ce7-4b35-ac45-09fb6e0e6c40 from DataBase >2014-09-11 06:59:24,918 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 36973cde >2014-09-11 06:59:24,918 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 06:59:24,918 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 06:59:24,989 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-46) [65c402b4] FINISH, SetVolumeDescriptionVDSCommand, log id: 36973cde >2014-09-11 06:59:25,031 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 06:59:25,033 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: fadb3dc >2014-09-11 06:59:25,033 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] -- executeVdsBrokerCommand: >2014-09-11 06:59:25,034 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 06:59:25,034 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 06:59:25,034 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 06:59:25,035 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] FINISH, UploadStreamVDSCommand, log id: fadb3dc >2014-09-11 06:59:25,035 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 06:59:25,045 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 06:59:25,045 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 06:59:25,045 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 06:59:25,045 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 06:59:25,045 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 06:59:25,053 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Removed task 784e277e-b7ed-417e-b899-ada7589c4fef from DataBase >2014-09-11 06:59:25,055 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 06:59:25,055 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 06:59:25,056 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 06:59:25,056 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-46) [26d77dff] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 06:59:25,056 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-46) [26d77dff] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 07:59:14,886 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) [34f87778] Principal admin::internal synchronized >2014-09-11 07:59:25,057 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) [92dad18] Attempting to update VMs/Templates Ovf. >2014-09-11 07:59:25,145 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 07:59:25,148 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 07:59:25,158 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 67e094df >2014-09-11 07:59:25,158 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 07:59:25,158 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 07:59:25,466 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [1d998d94] FINISH, SetVolumeDescriptionVDSCommand, log id: 67e094df >2014-09-11 07:59:25,522 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 07:59:25,524 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 4449b8ea >2014-09-11 07:59:25,524 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] -- executeVdsBrokerCommand: >2014-09-11 07:59:25,524 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 07:59:25,525 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 07:59:25,525 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 07:59:25,526 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] FINISH, UploadStreamVDSCommand, log id: 4449b8ea >2014-09-11 07:59:25,526 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 07:59:25,538 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 07:59:25,547 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] Removed task c59d4f77-0442-4925-90ef-67a1da246f4e from DataBase >2014-09-11 07:59:25,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 6173b8d1 >2014-09-11 07:59:25,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 07:59:25,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 07:59:25,781 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-93) [5f4f57d4] FINISH, SetVolumeDescriptionVDSCommand, log id: 6173b8d1 >2014-09-11 07:59:25,837 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 07:59:25,840 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 12ced236 >2014-09-11 07:59:25,840 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] -- executeVdsBrokerCommand: >2014-09-11 07:59:25,840 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 07:59:25,840 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 07:59:25,841 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 07:59:25,841 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] FINISH, UploadStreamVDSCommand, log id: 12ced236 >2014-09-11 07:59:25,841 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 07:59:25,857 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 07:59:25,857 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 07:59:25,858 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 07:59:25,858 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 07:59:25,858 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 07:59:25,873 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Removed task deb9020a-010e-48f8-8e41-9709b32776ad from DataBase >2014-09-11 07:59:25,875 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 07:59:25,875 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 07:59:25,875 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 07:59:25,875 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-93) [70d28647] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 07:59:25,875 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-93) [70d28647] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 08:59:14,897 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-11 08:59:25,876 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-09-11 08:59:25,933 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 08:59:25,936 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 08:59:25,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 6c07ec9b >2014-09-11 08:59:25,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 08:59:25,947 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 08:59:26,239 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [21e84658] FINISH, SetVolumeDescriptionVDSCommand, log id: 6c07ec9b >2014-09-11 08:59:26,289 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 08:59:26,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 6bc05f8a >2014-09-11 08:59:26,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] -- executeVdsBrokerCommand: >2014-09-11 08:59:26,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 08:59:26,291 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 08:59:26,291 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 08:59:26,291 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] FINISH, UploadStreamVDSCommand, log id: 6bc05f8a >2014-09-11 08:59:26,292 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 08:59:26,299 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 08:59:26,306 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-9) [792bd65e] Removed task 92892a25-31fe-4433-bbe3-159dfa290ca1 from DataBase >2014-09-11 08:59:26,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 48a03472 >2014-09-11 08:59:26,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 08:59:26,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 08:59:26,480 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-9) [792bd65e] FINISH, SetVolumeDescriptionVDSCommand, log id: 48a03472 >2014-09-11 08:59:26,518 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 08:59:26,521 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 5dcb3226 >2014-09-11 08:59:26,521 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] -- executeVdsBrokerCommand: >2014-09-11 08:59:26,521 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 08:59:26,522 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Failed in UploadStreamVDS method, for vds: rack; host: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 08:59:26,522 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Command UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper >2014-09-11 08:59:26,527 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] FINISH, UploadStreamVDSCommand, log id: 5dcb3226 >2014-09-11 08:59:26,528 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001) >2014-09-11 08:59:26,543 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 08:59:26,543 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 08:59:26,543 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 08:59:26,543 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 08:59:26,543 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 08:59:26,551 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Removed task 2ca42f19-888c-490a-b340-cec6ce844301 from DataBase >2014-09-11 08:59:26,553 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 08:59:26,553 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 08:59:26,553 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 08:59:26,553 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 08:59:26,554 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-9) [1813a2a2] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 09:18:04,467 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [40dd0204] Running command: LogoutUserCommand internal: false. >2014-09-11 09:18:04,470 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [71f6b1c8] Running command: LogoutUserCommand internal: false. >2014-09-11 09:18:04,474 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-11) [2d0773e6] Running command: LogoutUserCommand internal: false. >2014-09-11 09:18:04,478 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [4a2fac09] Running command: LogoutUserCommand internal: false. >2014-09-11 09:18:04,484 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [39224438] Running command: LogoutUserCommand internal: false. >2014-09-11 09:18:04,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [39224438] Correlation ID: 39224438, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 09:18:04,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [71f6b1c8] Correlation ID: 71f6b1c8, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 09:18:04,527 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [2d0773e6] Correlation ID: 2d0773e6, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 09:18:04,532 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [40dd0204] Correlation ID: 40dd0204, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 09:18:04,534 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [4a2fac09] Correlation ID: 4a2fac09, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 09:18:04,607 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-11 09:18:04,607 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-11 09:54:23,502 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-11 09:54:23,517 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-11 09:54:23,517 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-11 09:54:23,518 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-11 09:54:23,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-11 09:54:23,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-11 09:54:23,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-11 09:54:23,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-11 09:54:23,519 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-11 09:54:23,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-11 09:54:23,521 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-11 09:54:23,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-11 09:54:23,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-11 09:54:23,522 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-11 09:54:23,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-11 09:54:23,523 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-11 09:54:23,529 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-11 09:54:23,529 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-11 09:54:23,530 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-11 09:54:23,531 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-11 09:54:23,532 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-11 09:54:23,553 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-11 09:54:23,554 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-11 09:54:23,555 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-09-11 09:54:23,556 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-09-11 09:54:23,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-11 09:54:23,566 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-11 09:54:23,567 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-11 09:54:23,568 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-11 09:54:37,498 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 55) Start initializing Backend >2014-09-11 09:54:38,182 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 55) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-11 09:54:38,307 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 55) Running ovirt-engine 3.5.0-0.11.beta.el6ev >2014-09-11 09:54:38,307 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 55) Start initializing dictionaries >2014-09-11 09:54:38,311 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 55) Finished initializing dictionaries >2014-09-11 09:54:38,311 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Start initializing AuditLogCleanupManager >2014-09-11 09:54:38,311 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-11 09:54:38,336 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Finished initializing AuditLogCleanupManager >2014-09-11 09:54:38,336 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Start initializing CommandEntityCleanupManager >2014-09-11 09:54:38,337 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-11 09:54:38,339 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 55) Finished initializing CommandEntityCleanupManager >2014-09-11 09:54:38,341 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 55) Start initializing TagsDirector >2014-09-11 09:54:38,352 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 55) Tag root added to tree >2014-09-11 09:54:38,367 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 55) Finished initializing TagsDirector >2014-09-11 09:54:38,367 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 55) Start initializing IsoDomainListSyncronizer >2014-09-11 09:54:38,375 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 55) Finished initializing IsoDomainListSyncronizer >2014-09-11 09:54:38,424 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 55) Completed initializing handlers >2014-09-11 09:54:38,461 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 55) Start initializing ErrorTranslatorImpl >2014-09-11 09:54:38,481 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 55) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-11 09:54:38,482 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 55) Finished initializing ErrorTranslatorImpl >2014-09-11 09:54:38,482 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 55) Start initializing ErrorTranslatorImpl >2014-09-11 09:54:38,482 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 55) Finished initializing ErrorTranslatorImpl >2014-09-11 09:54:38,482 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 55) Mark incomplete jobs as UNKNOWN >2014-09-11 09:54:38,596 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 55) Start initializing JobRepositoryCleanupManager >2014-09-11 09:54:38,597 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 55) Finished initializing JobRepositoryCleanupManager >2014-09-11 09:54:38,597 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 55) Start initializing AutoRecoveryManager >2014-09-11 09:54:38,601 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 55) Finished initializing AutoRecoveryManager >2014-09-11 09:54:38,609 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 55) Start initializing ExecutionMessageDirector >2014-09-11 09:54:38,611 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 55) Finished initializing ExecutionMessageDirector >2014-09-11 09:54:38,750 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Loading extension 'builtin-authn-internal' >2014-09-11 09:54:38,752 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Extension 'builtin-authn-internal' loaded >2014-09-11 09:54:38,755 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Loading extension 'internal' >2014-09-11 09:54:38,760 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Extension 'internal' loaded >2014-09-11 09:54:38,766 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Initializing extension 'builtin-authn-internal' >2014-09-11 09:54:38,766 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Extension 'builtin-authn-internal' initialized >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Initializing extension 'internal' >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Extension 'internal' initialized >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Start of enabled extensions list >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-11 09:54:38,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 55) End of enabled extensions list >2014-09-11 09:54:39,121 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 54) Start initializing DbUserCacheManager >2014-09-11 09:54:39,132 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 54) Finished initializing DbUserCacheManager >2014-09-11 09:54:39,173 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 54) Initialization of AsyncTaskManager completed successfully. >2014-09-11 09:54:39,183 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) Start initializing ResourceManager >2014-09-11 09:54:39,568 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-11 09:54:39,744 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 54) Entered VdsManager constructor >2014-09-11 09:54:39,854 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 54) Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-11 09:54:40,232 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) VDS f689c28a-07de-4408-ac86-81382b6dd288 was added to the Resource Manager >2014-09-11 09:54:40,272 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 54) Finished initializing ResourceManager >2014-09-11 09:54:40,319 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 54) Initialization of OvfDataUpdater completed successfully. >2014-09-11 09:54:40,320 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Start scheduling to enable vds load balancer >2014-09-11 09:54:40,320 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Finished scheduling to enable vds load balancer >2014-09-11 09:54:40,321 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Start HA Reservation check >2014-09-11 09:54:40,323 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Finished HA Reservation check >2014-09-11 09:54:40,340 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-11 09:54:40,350 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 54) Init VM custom properties utilities >2014-09-11 09:54:40,351 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 54) Init device custom properties utilities >2014-09-11 09:54:40,371 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Initializing Scheduling manager >2014-09-11 09:54:40,408 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 64 >2014-09-11 09:54:40,488 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) External scheduler disabled, discovery skipped >2014-09-11 09:54:40,488 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 54) Initialized Scheduling manager >2014-09-11 09:54:40,489 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 54) Initializing DWH Heart Beat >2014-09-11 09:54:40,492 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 54) DWH Heart Beat initialized >2014-09-11 09:54:43,491 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-11 09:54:44,405 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288]), log id: 297b5780 >2014-09-11 09:54:44,425 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: 297b5780 >2014-09-11 09:54:45,151 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: rack >2014-09-11 09:54:45,483 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [18c02297] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-11 09:54:50,376 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-20) [3de201f6] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-11 09:54:50,402 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [3de201f6] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-11 09:54:50,447 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-11 09:54:50,447 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-11 09:54:50,448 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-11 09:54:50,448 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-11 09:54:50,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-11 09:54:50,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,449 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-11 09:54:50,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-11 09:54:50,451 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-11 09:54:51,188 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] Correlation ID: 3de201f6, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-11 09:54:52,062 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [3de201f6] hostFromVds::selectedVds - rack, spmStatus SPM, storage pool Default >2014-09-11 09:54:52,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [3de201f6] Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-11 09:54:52,111 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [3de201f6] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-11 09:54:52,121 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-11 09:54:52,167 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-8) [3de201f6] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 3587d5e5 >2014-09-11 09:54:52,311 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-8) [3de201f6] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-11 09:54:52,354 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-8) [3de201f6] START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 1f783805 >2014-09-11 09:54:52,366 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-8) [3de201f6] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1f783805 >2014-09-11 09:54:52,366 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-8) [3de201f6] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 3587d5e5 >2014-09-11 09:54:52,367 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-8) [3de201f6] Discovered no tasks on Storage Pool Default >2014-09-11 10:04:27,050 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-8) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-11 10:04:27,054 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-8) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-11 10:04:27,100 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-11 10:04:27,101 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-11 10:04:57,380 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginAdminUserCommand internal: false. >2014-09-11 10:04:57,468 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 10:05:00,690 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-09-11 10:05:21,343 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-11 10:05:21,349 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] Entered (VM f20). >2014-09-11 10:05:21,349 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] Sending shutdown command for VM f20. >2014-09-11 10:05:21,356 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 12bdacb >2014-09-11 10:05:21,366 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 25671fce >2014-09-11 10:05:21,372 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] FINISH, DestroyVDSCommand, log id: 25671fce >2014-09-11 10:05:21,396 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-46) [449a271] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 12bdacb >2014-09-11 10:05:21,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-46) [449a271] Correlation ID: 449a271, Job ID: 0be88845-d36a-4944-abe7-3c479989b6c4, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM f20 (Host: rack) (Reason: Not Specified). >2014-09-11 10:06:08,464 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [4ec1a21e] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@10.34.66.17' >2014-09-11 10:06:08,505 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [4ec1a21e] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-11 10:06:08,669 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:06:08,718 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-11 10:06:08,723 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] START, RemoveVdsVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 7777dbad >2014-09-11 10:06:08,725 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] FINISH, RemoveVdsVDSCommand, log id: 7777dbad >2014-09-11 10:06:08,730 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] START, AddVdsVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 65da46f2 >2014-09-11 10:06:08,730 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] AddVds - entered , starting logic to add VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f >2014-09-11 10:06:08,732 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] AddVds - VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f was added, will try to add it to the resource manager >2014-09-11 10:06:08,732 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [17cb9199] Entered VdsManager constructor >2014-09-11 10:06:08,751 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [17cb9199] Initialize vdsBroker (10.34.66.17,54,321) >2014-09-11 10:06:08,753 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [17cb9199] VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f was added to the Resource Manager >2014-09-11 10:06:08,753 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [17cb9199] FINISH, AddVdsVDSCommand, log id: 65da46f2 >2014-09-11 10:06:08,772 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [17cb9199] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host 10.34.66.17. >2014-09-11 10:06:08,839 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [17cb9199] Correlation ID: 4ec1a21e, Job ID: 0f4fd529-b94d-4678-9f4a-fd207177216c, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 was added by admin. >2014-09-11 10:06:08,842 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Lock Acquired to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:06:08,861 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:06:08,870 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Before Installation host 73e23c28-b882-49ee-946e-b4837bf1ec9f, 10.34.66.17 >2014-09-11 10:06:08,877 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 326a1f7a >2014-09-11 10:06:08,888 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] FINISH, SetVdsStatusVDSCommand, log id: 326a1f7a >2014-09-11 10:06:08,928 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Installation 10.34.66.17: Connected to host 10.34.66.17 with SSH key fingerprint: ae:88:e2:24:3b:64:66:39:1e:35:47:35:de:9e:d8:26 >2014-09-11 10:06:08,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Connected to host 10.34.66.17 with SSH key fingerprint: ae:88:e2:24:3b:64:66:39:1e:35:47:35:de:9e:d8:26. >2014-09-11 10:06:09,005 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Installation of 10.34.66.17. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-11 10:06:09,025 INFO [org.ovirt.engine.core.utils.archivers.tar.CachedTar] (org.ovirt.thread.pool-6-thread-20) Tarball '/var/cache/ovirt-engine/ovirt-host-deploy.tar' is out of date, re-creating >2014-09-11 10:06:09,189 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-20) SSH execute root@10.34.66.17 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-11 10:06:11,757 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-57) Initializing Host: 10.34.66.17 >2014-09-11 10:06:28,036 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Downloading Packages >2014-09-11 10:06:28,052 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Downloading Packages. >2014-09-11 10:06:28,957 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libselinux-python-2.0.94-5.3.el6_4.1.x86_64 >2014-09-11 10:06:28,972 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libselinux-python-2.0.94-5.3.el6_4.1.x86_64. >2014-09-11 10:06:28,972 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Check Package Signatures >2014-09-11 10:06:28,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Check Package Signatures. >2014-09-11 10:06:28,985 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Test Transaction >2014-09-11 10:06:28,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Test Transaction. >2014-09-11 10:06:30,085 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Transaction >2014-09-11 10:06:30,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Transaction. >2014-09-11 10:06:30,723 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 1/1: libselinux-python-2.0.94-5.3.el6_4.1.x86_64 >2014-09-11 10:06:30,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 1/1: libselinux-python-2.0.94-5.3.el6_4.1.x86_64. >2014-09-11 10:06:35,393 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.3.el6_4.1 - u >2014-09-11 10:06:35,406 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.3.el6_4.1 - u. >2014-09-11 10:06:36,159 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Initializing >2014-09-11 10:06:36,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Initializing. >2014-09-11 10:06:36,186 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Environment setup >2014-09-11 10:06:36,203 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Environment setup. >2014-09-11 10:06:36,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Environment packages setup >2014-09-11 10:06:36,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Environment packages setup. >2014-09-11 10:06:59,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Downloading Packages >2014-09-11 10:06:59,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Downloading Packages. >2014-09-11 10:07:00,816 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:dmidecode-2.12-5.el6_5.x86_64 >2014-09-11 10:07:00,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:dmidecode-2.12-5.el6_5.x86_64. >2014-09-11 10:07:02,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: iproute-2.6.32-32.el6_5.x86_64 >2014-09-11 10:07:02,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: iproute-2.6.32-32.el6_5.x86_64. >2014-09-11 10:07:02,238 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Check Package Signatures >2014-09-11 10:07:02,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Check Package Signatures. >2014-09-11 10:07:02,248 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Test Transaction >2014-09-11 10:07:02,266 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Test Transaction. >2014-09-11 10:07:02,547 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Transaction >2014-09-11 10:07:02,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Transaction. >2014-09-11 10:07:02,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 1/4: 1:dmidecode-2.12-5.el6_5.x86_64 >2014-09-11 10:07:02,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 1/4: 1:dmidecode-2.12-5.el6_5.x86_64. >2014-09-11 10:07:02,882 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 2/4: iproute-2.6.32-32.el6_5.x86_64 >2014-09-11 10:07:02,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 2/4: iproute-2.6.32-32.el6_5.x86_64. >2014-09-11 10:07:03,213 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 3/4: dmidecode >2014-09-11 10:07:03,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 3/4: dmidecode. >2014-09-11 10:07:03,270 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 4/4: iproute >2014-09-11 10:07:03,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 4/4: iproute. >2014-09-11 10:07:07,412 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 1/4: iproute.x86_64 0:2.6.32-32.el6_5 - u >2014-09-11 10:07:07,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 1/4: iproute.x86_64 0:2.6.32-32.el6_5 - u. >2014-09-11 10:07:07,428 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 2/4: dmidecode.x86_64 1:2.12-5.el6_5 - u >2014-09-11 10:07:07,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 2/4: dmidecode.x86_64 1:2.12-5.el6_5 - u. >2014-09-11 10:07:07,444 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 3/4: dmidecode.x86_64 1:2.11-2.el6 - ud >2014-09-11 10:07:07,454 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 3/4: dmidecode.x86_64 1:2.11-2.el6 - ud. >2014-09-11 10:07:07,455 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 4/4: iproute.x86_64 0:2.6.32-31.el6 - ud >2014-09-11 10:07:07,464 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 4/4: iproute.x86_64 0:2.6.32-31.el6 - ud. >2014-09-11 10:07:07,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Programs detection >2014-09-11 10:07:07,675 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Programs detection. >2014-09-11 10:07:07,676 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Environment customization >2014-09-11 10:07:07,685 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Environment customization. >2014-09-11 10:07:07,706 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Kdump unsupported >2014-09-11 10:07:07,716 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Kdump unsupported. >2014-09-11 10:07:07,742 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Logs at host located at: '/tmp/ovirt-host-deploy-20140911100609-59xv2s.log' >2014-09-11 10:07:07,752 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Logs at host located at: '/tmp/ovirt-host-deploy-20140911100609-59xv2s.log'. >2014-09-11 10:07:07,808 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host 10.34.66.17 reports unique id 27813927-2AC3-4778-A83B-3292123B096E >2014-09-11 10:07:07,887 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 27813927-2AC3-4778-A83B-3292123B096E to Host 10.34.66.17 >2014-09-11 10:07:08,011 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Disabling Kdump integration >2014-09-11 10:07:08,022 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Disabling Kdump integration. >2014-09-11 10:07:08,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Setup validation >2014-09-11 10:07:08,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Setup validation. >2014-09-11 10:07:08,069 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Cannot validate host name settings, reason: resolved host does not match any of the local addresses >2014-09-11 10:07:08,080 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 installation in progress . Cannot validate host name settings, reason: resolved host does not match any of the local addresses. >2014-09-11 10:07:08,080 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Machine does not support constant timestamp counter, this may effect performance >2014-09-11 10:07:08,090 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 installation in progress . Machine does not support constant timestamp counter, this may effect performance. >2014-09-11 10:07:08,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Hardware supports virtualization >2014-09-11 10:07:08,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Hardware supports virtualization. >2014-09-11 10:07:08,564 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Transaction setup >2014-09-11 10:07:08,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Transaction setup. >2014-09-11 10:07:08,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Misc configuration >2014-09-11 10:07:08,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Misc configuration. >2014-09-11 10:07:08,646 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Package installation >2014-09-11 10:07:08,656 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Package installation. >2014-09-11 10:07:39,100 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Downloading: rhel-x86_64-server-6-rhevm-3/filelists (0%) >2014-09-11 10:07:39,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Downloading: rhel-x86_64-server-6-rhevm-3/filelists (0%). >2014-09-11 10:08:01,481 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Downloading Packages >2014-09-11 10:08:01,496 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Downloading Packages. >2014-09-11 10:08:02,370 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: audit-2.2-4.el6_5.x86_64 >2014-09-11 10:08:02,384 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: audit-2.2-4.el6_5.x86_64. >2014-09-11 10:08:03,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: audit-libs-2.2-4.el6_5.x86_64 >2014-09-11 10:08:03,154 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: audit-libs-2.2-4.el6_5.x86_64. >2014-09-11 10:08:03,653 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: audit-libs-python-2.2-4.el6_5.x86_64 >2014-09-11 10:08:03,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: audit-libs-python-2.2-4.el6_5.x86_64. >2014-09-11 10:08:04,527 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: augeas-1.0.0-5.el6_5.1.x86_64 >2014-09-11 10:08:04,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: augeas-1.0.0-5.el6_5.1.x86_64. >2014-09-11 10:08:05,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: augeas-libs-1.0.0-5.el6_5.1.x86_64 >2014-09-11 10:08:05,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: augeas-libs-1.0.0-5.el6_5.1.x86_64. >2014-09-11 10:08:06,718 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64 >2014-09-11 10:08:06,729 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64. >2014-09-11 10:08:07,612 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: celt051-0.5.1.3-0.el6.x86_64 >2014-09-11 10:08:07,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: celt051-0.5.1.3-0.el6.x86_64. >2014-09-11 10:08:09,896 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: cvs-1.11.23-16.el6.x86_64 >2014-09-11 10:08:09,907 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: cvs-1.11.23-16.el6.x86_64. >2014-09-11 10:08:10,682 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 >2014-09-11 10:08:10,697 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64. >2014-09-11 10:08:11,617 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:08:11,647 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:08:12,426 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:08:12,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:08:13,475 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: dnsmasq-2.48-13.el6.x86_64 >2014-09-11 10:08:13,489 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: dnsmasq-2.48-13.el6.x86_64. >2014-09-11 10:08:14,364 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: ebtables-2.0.9-6.el6.x86_64 >2014-09-11 10:08:14,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: ebtables-2.0.9-6.el6.x86_64. >2014-09-11 10:08:15,404 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: febootstrap-supermin-helper-3.21-4.el6.x86_64 >2014-09-11 10:08:15,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: febootstrap-supermin-helper-3.21-4.el6.x86_64. >2014-09-11 10:08:16,302 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: fence-agents-3.1.5-35.el6_5.4.x86_64 >2014-09-11 10:08:16,319 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: fence-agents-3.1.5-35.el6_5.4.x86_64. >2014-09-11 10:08:17,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: flac-1.2.1-6.1.el6.x86_64 >2014-09-11 10:08:17,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: flac-1.2.1-6.1.el6.x86_64. >2014-09-11 10:08:18,565 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: fuse-2.8.3-4.el6.x86_64 >2014-09-11 10:08:18,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: fuse-2.8.3-4.el6.x86_64. >2014-09-11 10:08:19,448 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: fuse-libs-2.8.3-4.el6.x86_64 >2014-09-11 10:08:19,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: fuse-libs-2.8.3-4.el6.x86_64. >2014-09-11 10:08:20,354 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: genisoimage-1.1.9-12.el6.x86_64 >2014-09-11 10:08:20,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: genisoimage-1.1.9-12.el6.x86_64. >2014-09-11 10:08:23,656 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: gettext-0.17-16.el6.x86_64 >2014-09-11 10:08:23,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: gettext-0.17-16.el6.x86_64. >2014-09-11 10:08:29,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: ghostscript-8.70-19.el6.x86_64 >2014-09-11 10:08:29,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: ghostscript-8.70-19.el6.x86_64. >2014-09-11 10:08:31,422 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: ghostscript-fonts-5.50-23.1.el6.noarch >2014-09-11 10:08:31,437 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: ghostscript-fonts-5.50-23.1.el6.noarch. >2014-09-11 10:08:35,442 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: glusterfs-3.5.2-1.el6.x86_64 >2014-09-11 10:08:35,453 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: glusterfs-3.5.2-1.el6.x86_64. >2014-09-11 10:08:36,789 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: glusterfs-api-3.5.2-1.el6.x86_64 >2014-09-11 10:08:36,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: glusterfs-api-3.5.2-1.el6.x86_64. >2014-09-11 10:08:39,329 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: glusterfs-libs-3.5.2-1.el6.x86_64 >2014-09-11 10:08:39,343 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: glusterfs-libs-3.5.2-1.el6.x86_64. >2014-09-11 10:08:40,601 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: gnutls-2.8.5-14.el6_5.x86_64 >2014-09-11 10:08:40,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: gnutls-2.8.5-14.el6_5.x86_64. >2014-09-11 10:08:41,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: gnutls-utils-2.8.5-14.el6_5.x86_64 >2014-09-11 10:08:41,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: gnutls-utils-2.8.5-14.el6_5.x86_64. >2014-09-11 10:08:42,491 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: gpxe-roms-qemu-0.9.7-6.10.el6.noarch >2014-09-11 10:08:42,501 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: gpxe-roms-qemu-0.9.7-6.10.el6.noarch. >2014-09-11 10:08:43,588 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: hivex-1.3.3-4.2.el6.x86_64 >2014-09-11 10:08:43,598 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: hivex-1.3.3-4.2.el6.x86_64. >2014-09-11 10:08:43,606 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: ioprocess-0.12.0-2.el6ev.x86_64 >2014-09-11 10:08:43,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: ioprocess-0.12.0-2.el6ev.x86_64. >2014-09-11 10:08:44,683 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: ipmitool-1.8.11-16.el6.x86_64 >2014-09-11 10:08:44,699 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: ipmitool-1.8.11-16.el6.x86_64. >2014-09-11 10:08:45,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: keyutils-1.4-4.el6.x86_64 >2014-09-11 10:08:45,426 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: keyutils-1.4-4.el6.x86_64. >2014-09-11 10:08:46,428 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: kpartx-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:08:46,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: kpartx-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:08:47,181 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libXfont-1.4.5-3.el6_5.x86_64 >2014-09-11 10:08:47,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libXfont-1.4.5-3.el6_5.x86_64. >2014-09-11 10:08:48,004 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libXtst-1.2.1-2.el6.x86_64 >2014-09-11 10:08:48,014 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libXtst-1.2.1-2.el6.x86_64. >2014-09-11 10:08:48,618 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libasyncns-0.8-1.1.el6.x86_64 >2014-09-11 10:08:48,631 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libasyncns-0.8-1.1.el6.x86_64. >2014-09-11 10:08:49,284 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libcgroup-0.40.rc1-5.el6_5.1.x86_64 >2014-09-11 10:08:49,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libcgroup-0.40.rc1-5.el6_5.1.x86_64. >2014-09-11 10:08:50,037 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libconfig-1.3.2-1.1.el6.x86_64 >2014-09-11 10:08:50,046 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libconfig-1.3.2-1.1.el6.x86_64. >2014-09-11 10:08:50,739 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libevent-1.4.13-4.el6.x86_64 >2014-09-11 10:08:50,750 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libevent-1.4.13-4.el6.x86_64. >2014-09-11 10:08:51,591 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libfontenc-1.0.5-2.el6.x86_64 >2014-09-11 10:08:51,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libfontenc-1.0.5-2.el6.x86_64. >2014-09-11 10:08:52,379 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libgomp-4.4.7-4.el6.x86_64 >2014-09-11 10:08:52,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libgomp-4.4.7-4.el6.x86_64. >2014-09-11 10:08:53,628 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libgssglue-0.1-11.el6.x86_64 >2014-09-11 10:08:53,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libgssglue-0.1-11.el6.x86_64. >2014-09-11 10:08:57,170 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:libguestfs-1.20.11-2.el6.x86_64 >2014-09-11 10:08:57,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:libguestfs-1.20.11-2.el6.x86_64. >2014-09-11 10:08:59,450 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64 >2014-09-11 10:08:59,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64. >2014-09-11 10:09:00,056 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 2:libogg-1.1.4-2.1.el6.x86_64 >2014-09-11 10:09:00,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 2:libogg-1.1.4-2.1.el6.x86_64. >2014-09-11 10:09:00,753 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libsemanage-python-2.0.43-4.2.el6.x86_64 >2014-09-11 10:09:00,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libsemanage-python-2.0.43-4.2.el6.x86_64. >2014-09-11 10:09:01,596 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libsndfile-1.0.20-5.el6.x86_64 >2014-09-11 10:09:01,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libsndfile-1.0.20-5.el6.x86_64. >2014-09-11 10:09:02,355 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libtirpc-0.2.1-6.el6_5.2.x86_64 >2014-09-11 10:09:02,367 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libtirpc-0.2.1-6.el6_5.2.x86_64. >2014-09-11 10:09:05,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libvirt-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:09:05,539 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libvirt-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:09:09,021 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libvirt-client-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:09:09,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libvirt-client-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:09:09,773 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:09:09,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:09:11,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: libvirt-python-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:09:11,101 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: libvirt-python-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:09:11,928 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:libvorbis-1.2.3-4.el6_2.1.x86_64 >2014-09-11 10:09:11,942 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:libvorbis-1.2.3-4.el6_2.1.x86_64. >2014-09-11 10:09:12,696 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: lm_sensors-libs-3.1.1-17.el6.x86_64 >2014-09-11 10:09:12,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: lm_sensors-libs-3.1.1-17.el6.x86_64. >2014-09-11 10:09:12,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: lzop-1.02-0.9.rc1.el6.x86_64 >2014-09-11 10:09:12,730 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: lzop-1.02-0.9.rc1.el6.x86_64. >2014-09-11 10:09:12,744 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: mom-0.4.1-2.el6.noarch >2014-09-11 10:09:12,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: mom-0.4.1-2.el6.noarch. >2014-09-11 10:09:13,459 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: nc-1.84-22.el6.x86_64 >2014-09-11 10:09:13,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: nc-1.84-22.el6.x86_64. >2014-09-11 10:09:14,993 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64 >2014-09-11 10:09:15,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64. >2014-09-11 10:09:15,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64 >2014-09-11 10:09:15,758 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64. >2014-09-11 10:09:16,621 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: netcf-libs-0.1.9-4.el6_5.2.x86_64 >2014-09-11 10:09:16,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: netcf-libs-0.1.9-4.el6_5.2.x86_64. >2014-09-11 10:09:18,697 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: netpbm-10.47.05-11.el6.x86_64 >2014-09-11 10:09:18,707 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: netpbm-10.47.05-11.el6.x86_64. >2014-09-11 10:09:23,683 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: netpbm-progs-10.47.05-11.el6.x86_64 >2014-09-11 10:09:23,695 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: netpbm-progs-10.47.05-11.el6.x86_64. >2014-09-11 10:09:24,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64 >2014-09-11 10:09:24,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64. >2014-09-11 10:09:25,578 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: nfs-utils-lib-1.1.5-6.el6_5.x86_64 >2014-09-11 10:09:25,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: nfs-utils-lib-1.1.5-6.el6_5.x86_64. >2014-09-11 10:09:26,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: numad-0.5-10.20140620git.el6_5.x86_64 >2014-09-11 10:09:26,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: numad-0.5-10.20140620git.el6_5.x86_64. >2014-09-11 10:09:27,233 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: perl-Net-Telnet-3.03-11.el6.noarch >2014-09-11 10:09:27,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: perl-Net-Telnet-3.03-11.el6.noarch. >2014-09-11 10:09:28,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: perl-hivex-1.3.3-4.2.el6.x86_64 >2014-09-11 10:09:28,116 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: perl-hivex-1.3.3-4.2.el6.x86_64. >2014-09-11 10:09:29,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: pexpect-2.3-6.el6.noarch >2014-09-11 10:09:29,091 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: pexpect-2.3-6.el6.noarch. >2014-09-11 10:09:30,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: policycoreutils-python-2.0.83-19.39.el6.x86_64 >2014-09-11 10:09:30,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: policycoreutils-python-2.0.83-19.39.el6.x86_64. >2014-09-11 10:09:31,507 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: pulseaudio-libs-0.9.21-14.el6_3.x86_64 >2014-09-11 10:09:31,522 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: pulseaudio-libs-0.9.21-14.el6_3.x86_64. >2014-09-11 10:09:32,621 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: pyparted-3.4-4.el6.x86_64 >2014-09-11 10:09:32,634 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: pyparted-3.4-4.el6.x86_64. >2014-09-11 10:09:32,744 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-argparse-1.2.1-2.1.el6.noarch >2014-09-11 10:09:32,762 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-argparse-1.2.1-2.1.el6.noarch. >2014-09-11 10:09:32,764 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-cpopen-1.3-4.el6_5.x86_64 >2014-09-11 10:09:32,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-cpopen-1.3-4.el6_5.x86_64. >2014-09-11 10:09:33,388 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-decorator-3.0.1-3.1.el6.noarch >2014-09-11 10:09:33,411 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-decorator-3.0.1-3.1.el6.noarch. >2014-09-11 10:09:33,468 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-inotify-0.9.1-1.el6ev.noarch >2014-09-11 10:09:33,481 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-inotify-0.9.1-1.el6ev.noarch. >2014-09-11 10:09:33,482 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-ioprocess-0.12.0-2.el6ev.noarch >2014-09-11 10:09:33,494 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-ioprocess-0.12.0-2.el6ev.noarch. >2014-09-11 10:09:36,078 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-netaddr-0.7.5-4.el6.noarch >2014-09-11 10:09:36,091 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-netaddr-0.7.5-4.el6.noarch. >2014-09-11 10:09:36,135 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-ordereddict-1.1-3.el6ev.noarch >2014-09-11 10:09:36,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-ordereddict-1.1-3.el6ev.noarch. >2014-09-11 10:09:36,230 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-pthreading-0.1.3-3.el6ev.noarch >2014-09-11 10:09:36,250 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-pthreading-0.1.3-3.el6ev.noarch. >2014-09-11 10:09:37,509 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: python-suds-0.4.1-3.el6.noarch >2014-09-11 10:09:37,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: python-suds-0.4.1-3.el6.noarch. >2014-09-11 10:09:37,585 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:09:37,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:09:37,677 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:09:37,713 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:09:37,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:09:37,783 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:09:38,750 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: radvd-1.6-1.el6.x86_64 >2014-09-11 10:09:38,764 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: radvd-1.6-1.el6.x86_64. >2014-09-11 10:09:39,555 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: rpcbind-0.2.0-11.el6.x86_64 >2014-09-11 10:09:39,566 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: rpcbind-0.2.0-11.el6.x86_64. >2014-09-11 10:09:40,552 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: sanlock-2.8-1.el6.x86_64 >2014-09-11 10:09:40,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: sanlock-2.8-1.el6.x86_64. >2014-09-11 10:09:41,175 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: sanlock-lib-2.8-1.el6.x86_64 >2014-09-11 10:09:41,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: sanlock-lib-2.8-1.el6.x86_64. >2014-09-11 10:09:42,096 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: sanlock-python-2.8-1.el6.x86_64 >2014-09-11 10:09:42,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: sanlock-python-2.8-1.el6.x86_64. >2014-09-11 10:09:42,984 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: scrub-2.2-1.el6.x86_64 >2014-09-11 10:09:42,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: scrub-2.2-1.el6.x86_64. >2014-09-11 10:09:43,956 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: seabios-0.6.1.2-28.el6.x86_64 >2014-09-11 10:09:43,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: seabios-0.6.1.2-28.el6.x86_64. >2014-09-11 10:09:45,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: setools-libs-3.3.7-4.el6.x86_64 >2014-09-11 10:09:45,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: setools-libs-3.3.7-4.el6.x86_64. >2014-09-11 10:09:46,154 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: setools-libs-python-3.3.7-4.el6.x86_64 >2014-09-11 10:09:46,167 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: setools-libs-python-3.3.7-4.el6.x86_64. >2014-09-11 10:09:47,797 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: sg3_utils-1.28-5.el6.x86_64 >2014-09-11 10:09:47,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: sg3_utils-1.28-5.el6.x86_64. >2014-09-11 10:09:48,329 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: sgabios-bin-0-0.3.20110621svn.el6.noarch >2014-09-11 10:09:48,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: sgabios-bin-0-0.3.20110621svn.el6.noarch. >2014-09-11 10:09:49,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: spice-server-0.12.4-6.el6_5.2.x86_64 >2014-09-11 10:09:49,393 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: spice-server-0.12.4-6.el6_5.2.x86_64. >2014-09-11 10:09:50,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:telnet-0.17-47.el6_3.1.x86_64 >2014-09-11 10:09:50,296 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:telnet-0.17-47.el6_3.1.x86_64. >2014-09-11 10:09:51,125 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: tree-1.5.3-2.el6.x86_64 >2014-09-11 10:09:51,137 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: tree-1.5.3-2.el6.x86_64. >2014-09-11 10:09:58,197 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: urw-fonts-2.4-10.el6.noarch >2014-09-11 10:09:58,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: urw-fonts-2.4-10.el6.noarch. >2014-09-11 10:09:58,954 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: usbredir-0.5.1-1.el6.x86_64 >2014-09-11 10:09:58,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: usbredir-0.5.1-1.el6.x86_64. >2014-09-11 10:09:58,986 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-4.16.3-3.el6ev.beta.x86_64 >2014-09-11 10:09:59,001 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-4.16.3-3.el6ev.beta.x86_64. >2014-09-11 10:09:59,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-cli-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,014 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-cli-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,014 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-jsonrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-jsonrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,026 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-python-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-python-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,039 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,050 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,050 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-xmlrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,060 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-xmlrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,060 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vdsm-yajsonrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:09:59,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vdsm-yajsonrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:09:59,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: vgabios-0.6b-3.7.el6.noarch >2014-09-11 10:09:59,817 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: vgabios-0.6b-3.7.el6.noarch. >2014-09-11 10:10:00,537 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: 1:xorg-x11-font-utils-7.2-11.el6.x86_64 >2014-09-11 10:10:00,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: 1:xorg-x11-font-utils-7.2-11.el6.x86_64. >2014-09-11 10:10:01,299 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Download/Verify: yajl-1.0.7-3.el6.x86_64 >2014-09-11 10:10:01,317 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Download/Verify: yajl-1.0.7-3.el6.x86_64. >2014-09-11 10:10:01,317 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Check Package Signatures >2014-09-11 10:10:01,336 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Check Package Signatures. >2014-09-11 10:10:01,351 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Confirm use of GPG Key userid=Red Hat, Inc. (release key 2) <security@redhat.com> hexkeyid=FD431D51 >2014-09-11 10:10:01,365 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 installation in progress . Confirm use of GPG Key userid=Red Hat, Inc. (release key 2) <security@redhat.com> hexkeyid=FD431D51. >2014-09-11 10:10:01,552 WARN [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Confirm use of GPG Key userid=Red Hat, Inc. (auxiliary key) <security@redhat.com> hexkeyid=2FA658E0 >2014-09-11 10:10:01,564 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 installation in progress . Confirm use of GPG Key userid=Red Hat, Inc. (auxiliary key) <security@redhat.com> hexkeyid=2FA658E0. >2014-09-11 10:10:01,610 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Test Transaction >2014-09-11 10:10:01,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Test Transaction. >2014-09-11 10:10:03,210 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Status: Running Transaction >2014-09-11 10:10:03,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Status: Running Transaction. >2014-09-11 10:10:03,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 1/113: audit-libs-2.2-4.el6_5.x86_64 >2014-09-11 10:10:03,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 1/113: audit-libs-2.2-4.el6_5.x86_64. >2014-09-11 10:10:06,802 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 2/113: yajl-1.0.7-3.el6.x86_64 >2014-09-11 10:10:06,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 2/113: yajl-1.0.7-3.el6.x86_64. >2014-09-11 10:10:06,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 3/113: gnutls-2.8.5-14.el6_5.x86_64 >2014-09-11 10:10:06,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 3/113: gnutls-2.8.5-14.el6_5.x86_64. >2014-09-11 10:10:07,097 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 4/113: libgssglue-0.1-11.el6.x86_64 >2014-09-11 10:10:07,104 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 4/113: libgssglue-0.1-11.el6.x86_64. >2014-09-11 10:10:07,197 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 5/113: glusterfs-libs-3.5.2-1.el6.x86_64 >2014-09-11 10:10:07,204 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 5/113: glusterfs-libs-3.5.2-1.el6.x86_64. >2014-09-11 10:10:07,321 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 6/113: 2:libogg-1.1.4-2.1.el6.x86_64 >2014-09-11 10:10:07,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 6/113: 2:libogg-1.1.4-2.1.el6.x86_64. >2014-09-11 10:10:07,424 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 7/113: augeas-libs-1.0.0-5.el6_5.1.x86_64 >2014-09-11 10:10:07,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 7/113: augeas-libs-1.0.0-5.el6_5.1.x86_64. >2014-09-11 10:10:08,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 8/113: sanlock-lib-2.8-1.el6.x86_64 >2014-09-11 10:10:08,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 8/113: sanlock-lib-2.8-1.el6.x86_64. >2014-09-11 10:10:08,337 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 9/113: libcgroup-0.40.rc1-5.el6_5.1.x86_64 >2014-09-11 10:10:08,343 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 9/113: libcgroup-0.40.rc1-5.el6_5.1.x86_64. >2014-09-11 10:10:08,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 10/113: python-cpopen-1.3-4.el6_5.x86_64 >2014-09-11 10:10:08,884 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 10/113: python-cpopen-1.3-4.el6_5.x86_64. >2014-09-11 10:10:09,218 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 11/113: sanlock-2.8-1.el6.x86_64 >2014-09-11 10:10:09,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 11/113: sanlock-2.8-1.el6.x86_64. >2014-09-11 10:10:09,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 12/113: libtirpc-0.2.1-6.el6_5.2.x86_64 >2014-09-11 10:10:09,374 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 12/113: libtirpc-0.2.1-6.el6_5.2.x86_64. >2014-09-11 10:10:09,457 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 13/113: perl-hivex-1.3.3-4.2.el6.x86_64 >2014-09-11 10:10:09,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 13/113: perl-hivex-1.3.3-4.2.el6.x86_64. >2014-09-11 10:10:09,548 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 14/113: hivex-1.3.3-4.2.el6.x86_64 >2014-09-11 10:10:09,557 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 14/113: hivex-1.3.3-4.2.el6.x86_64. >2014-09-11 10:10:09,687 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 15/113: lzop-1.02-0.9.rc1.el6.x86_64 >2014-09-11 10:10:09,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 15/113: lzop-1.02-0.9.rc1.el6.x86_64. >2014-09-11 10:10:09,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 16/113: libfontenc-1.0.5-2.el6.x86_64 >2014-09-11 10:10:09,765 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 16/113: libfontenc-1.0.5-2.el6.x86_64. >2014-09-11 10:10:09,872 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 17/113: fuse-libs-2.8.3-4.el6.x86_64 >2014-09-11 10:10:09,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 17/113: fuse-libs-2.8.3-4.el6.x86_64. >2014-09-11 10:10:10,201 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 18/113: usbredir-0.5.1-1.el6.x86_64 >2014-09-11 10:10:10,210 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 18/113: usbredir-0.5.1-1.el6.x86_64. >2014-09-11 10:10:10,352 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 19/113: vdsm-python-zombiereaper-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:10,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 19/113: vdsm-python-zombiereaper-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:10,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 20/113: libXfont-1.4.5-3.el6_5.x86_64 >2014-09-11 10:10:10,413 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 20/113: libXfont-1.4.5-3.el6_5.x86_64. >2014-09-11 10:10:10,504 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 21/113: 1:xorg-x11-font-utils-7.2-11.el6.x86_64 >2014-09-11 10:10:10,513 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 21/113: 1:xorg-x11-font-utils-7.2-11.el6.x86_64. >2014-09-11 10:10:10,750 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 22/113: ghostscript-fonts-5.50-23.1.el6.noarch >2014-09-11 10:10:10,759 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 22/113: ghostscript-fonts-5.50-23.1.el6.noarch. >2014-09-11 10:10:13,184 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 23/113: rpcbind-0.2.0-11.el6.x86_64 >2014-09-11 10:10:13,195 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 23/113: rpcbind-0.2.0-11.el6.x86_64. >2014-09-11 10:10:13,271 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 24/113: numad-0.5-10.20140620git.el6_5.x86_64 >2014-09-11 10:10:13,280 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 24/113: numad-0.5-10.20140620git.el6_5.x86_64. >2014-09-11 10:10:13,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 25/113: sanlock-python-2.8-1.el6.x86_64 >2014-09-11 10:10:13,387 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 25/113: sanlock-python-2.8-1.el6.x86_64. >2014-09-11 10:10:13,545 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 26/113: augeas-1.0.0-5.el6_5.1.x86_64 >2014-09-11 10:10:13,555 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 26/113: augeas-1.0.0-5.el6_5.1.x86_64. >2014-09-11 10:10:13,971 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 27/113: netcf-libs-0.1.9-4.el6_5.2.x86_64 >2014-09-11 10:10:13,981 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 27/113: netcf-libs-0.1.9-4.el6_5.2.x86_64. >2014-09-11 10:10:14,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 28/113: celt051-0.5.1.3-0.el6.x86_64 >2014-09-11 10:10:14,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 28/113: celt051-0.5.1.3-0.el6.x86_64. >2014-09-11 10:10:14,281 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 29/113: spice-server-0.12.4-6.el6_5.2.x86_64 >2014-09-11 10:10:14,291 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 29/113: spice-server-0.12.4-6.el6_5.2.x86_64. >2014-09-11 10:10:14,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 30/113: flac-1.2.1-6.1.el6.x86_64 >2014-09-11 10:10:14,442 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 30/113: flac-1.2.1-6.1.el6.x86_64. >2014-09-11 10:10:14,565 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 31/113: 1:libvorbis-1.2.3-4.el6_2.1.x86_64 >2014-09-11 10:10:14,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 31/113: 1:libvorbis-1.2.3-4.el6_2.1.x86_64. >2014-09-11 10:10:14,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 32/113: libsndfile-1.0.20-5.el6.x86_64 >2014-09-11 10:10:14,812 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 32/113: libsndfile-1.0.20-5.el6.x86_64. >2014-09-11 10:10:15,137 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 33/113: glusterfs-3.5.2-1.el6.x86_64 >2014-09-11 10:10:15,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 33/113: glusterfs-3.5.2-1.el6.x86_64. >2014-09-11 10:10:15,463 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 34/113: glusterfs-api-3.5.2-1.el6.x86_64 >2014-09-11 10:10:15,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 34/113: glusterfs-api-3.5.2-1.el6.x86_64. >2014-09-11 10:10:15,573 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 35/113: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:10:15,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 35/113: 2:qemu-img-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:10:15,696 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 36/113: gnutls-utils-2.8.5-14.el6_5.x86_64 >2014-09-11 10:10:15,707 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 36/113: gnutls-utils-2.8.5-14.el6_5.x86_64. >2014-09-11 10:10:15,770 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 37/113: ioprocess-0.12.0-2.el6ev.x86_64 >2014-09-11 10:10:15,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 37/113: ioprocess-0.12.0-2.el6ev.x86_64. >2014-09-11 10:10:15,834 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 38/113: python-ioprocess-0.12.0-2.el6ev.noarch >2014-09-11 10:10:15,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 38/113: python-ioprocess-0.12.0-2.el6ev.noarch. >2014-09-11 10:10:15,897 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 39/113: vdsm-python-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:15,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 39/113: vdsm-python-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:16,081 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 40/113: vdsm-xmlrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:16,092 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 40/113: vdsm-xmlrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:16,134 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 41/113: vdsm-cli-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:16,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 41/113: vdsm-cli-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:16,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 42/113: audit-libs-python-2.2-4.el6_5.x86_64 >2014-09-11 10:10:16,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 42/113: audit-libs-python-2.2-4.el6_5.x86_64. >2014-09-11 10:10:16,493 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 43/113: lm_sensors-libs-3.1.1-17.el6.x86_64 >2014-09-11 10:10:16,504 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 43/113: lm_sensors-libs-3.1.1-17.el6.x86_64. >2014-09-11 10:10:16,584 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 44/113: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64 >2014-09-11 10:10:16,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 44/113: 1:net-snmp-libs-5.5-49.el6_5.2.x86_64. >2014-09-11 10:10:16,959 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 45/113: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64 >2014-09-11 10:10:16,970 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 45/113: 1:net-snmp-utils-5.5-49.el6_5.2.x86_64. >2014-09-11 10:10:17,111 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 46/113: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64 >2014-09-11 10:10:17,122 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 46/113: cyrus-sasl-md5-2.1.23-13.el6_3.1.x86_64. >2014-09-11 10:10:17,241 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 47/113: ipmitool-1.8.11-16.el6.x86_64 >2014-09-11 10:10:17,251 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 47/113: ipmitool-1.8.11-16.el6.x86_64. >2014-09-11 10:10:17,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 48/113: libgomp-4.4.7-4.el6.x86_64 >2014-09-11 10:10:17,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 48/113: libgomp-4.4.7-4.el6.x86_64. >2014-09-11 10:10:17,770 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 49/113: python-netaddr-0.7.5-4.el6.noarch >2014-09-11 10:10:17,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 49/113: python-netaddr-0.7.5-4.el6.noarch. >2014-09-11 10:10:18,133 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 50/113: dnsmasq-2.48-13.el6.x86_64 >2014-09-11 10:10:18,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 50/113: dnsmasq-2.48-13.el6.x86_64. >2014-09-11 10:10:18,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 51/113: python-ordereddict-1.1-3.el6ev.noarch >2014-09-11 10:10:18,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 51/113: python-ordereddict-1.1-3.el6ev.noarch. >2014-09-11 10:10:18,312 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 52/113: sg3_utils-1.28-5.el6.x86_64 >2014-09-11 10:10:18,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 52/113: sg3_utils-1.28-5.el6.x86_64. >2014-09-11 10:10:18,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 53/113: python-argparse-1.2.1-2.1.el6.noarch >2014-09-11 10:10:18,641 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 53/113: python-argparse-1.2.1-2.1.el6.noarch. >2014-09-11 10:10:18,699 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 54/113: cvs-1.11.23-16.el6.x86_64 >2014-09-11 10:10:18,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 54/113: cvs-1.11.23-16.el6.x86_64. >2014-09-11 10:10:18,892 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 55/113: gettext-0.17-16.el6.x86_64 >2014-09-11 10:10:18,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 55/113: gettext-0.17-16.el6.x86_64. >2014-09-11 10:10:19,589 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 56/113: perl-Net-Telnet-3.03-11.el6.noarch >2014-09-11 10:10:19,599 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 56/113: perl-Net-Telnet-3.03-11.el6.noarch. >2014-09-11 10:10:19,680 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 57/113: libXtst-1.2.1-2.el6.x86_64 >2014-09-11 10:10:19,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 57/113: libXtst-1.2.1-2.el6.x86_64. >2014-09-11 10:10:19,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 58/113: seabios-0.6.1.2-28.el6.x86_64 >2014-09-11 10:10:19,780 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 58/113: seabios-0.6.1.2-28.el6.x86_64. >2014-09-11 10:10:19,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 59/113: python-suds-0.4.1-3.el6.noarch >2014-09-11 10:10:19,842 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 59/113: python-suds-0.4.1-3.el6.noarch. >2014-09-11 10:10:20,067 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 60/113: 1:telnet-0.17-47.el6_3.1.x86_64 >2014-09-11 10:10:20,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 60/113: 1:telnet-0.17-47.el6_3.1.x86_64. >2014-09-11 10:10:20,141 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 61/113: urw-fonts-2.4-10.el6.noarch >2014-09-11 10:10:20,151 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 61/113: urw-fonts-2.4-10.el6.noarch. >2014-09-11 10:10:21,390 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-3) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from PoweringDown --> Up >2014-09-11 10:10:21,415 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Shutdown of VM f20 failed. >2014-09-11 10:10:22,679 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 62/113: ghostscript-8.70-19.el6.x86_64 >2014-09-11 10:10:22,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 62/113: ghostscript-8.70-19.el6.x86_64. >2014-09-11 10:10:24,142 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 63/113: libevent-1.4.13-4.el6.x86_64 >2014-09-11 10:10:24,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 63/113: libevent-1.4.13-4.el6.x86_64. >2014-09-11 10:10:24,344 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 64/113: python-inotify-0.9.1-1.el6ev.noarch >2014-09-11 10:10:24,354 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 64/113: python-inotify-0.9.1-1.el6ev.noarch. >2014-09-11 10:10:25,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 65/113: ebtables-2.0.9-6.el6.x86_64 >2014-09-11 10:10:25,058 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 65/113: ebtables-2.0.9-6.el6.x86_64. >2014-09-11 10:10:25,202 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 66/113: pexpect-2.3-6.el6.noarch >2014-09-11 10:10:25,212 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 66/113: pexpect-2.3-6.el6.noarch. >2014-09-11 10:10:25,360 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 67/113: keyutils-1.4-4.el6.x86_64 >2014-09-11 10:10:25,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 67/113: keyutils-1.4-4.el6.x86_64. >2014-09-11 10:10:25,471 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 68/113: nfs-utils-lib-1.1.5-6.el6_5.x86_64 >2014-09-11 10:10:25,479 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 68/113: nfs-utils-lib-1.1.5-6.el6_5.x86_64. >2014-09-11 10:10:25,957 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 69/113: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64 >2014-09-11 10:10:25,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 69/113: 1:nfs-utils-1.2.3-39.el6_5.3.x86_64. >2014-09-11 10:10:26,312 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 70/113: vdsm-yajsonrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:26,321 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 70/113: vdsm-yajsonrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:26,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 71/113: vdsm-jsonrpc-4.16.3-3.el6ev.beta.noarch >2014-09-11 10:10:26,439 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 71/113: vdsm-jsonrpc-4.16.3-3.el6ev.beta.noarch. >2014-09-11 10:10:26,522 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 72/113: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64 >2014-09-11 10:10:26,531 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 72/113: btrfs-progs-0.20-0.2.git91d9eec.el6.x86_64. >2014-09-11 10:10:26,659 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 73/113: genisoimage-1.1.9-12.el6.x86_64 >2014-09-11 10:10:26,668 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 73/113: genisoimage-1.1.9-12.el6.x86_64. >2014-09-11 10:10:26,994 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 74/113: scrub-2.2-1.el6.x86_64 >2014-09-11 10:10:27,003 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 74/113: scrub-2.2-1.el6.x86_64. >2014-09-11 10:10:27,067 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 75/113: setools-libs-3.3.7-4.el6.x86_64 >2014-09-11 10:10:27,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 75/113: setools-libs-3.3.7-4.el6.x86_64. >2014-09-11 10:10:27,375 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 76/113: setools-libs-python-3.3.7-4.el6.x86_64 >2014-09-11 10:10:27,390 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 76/113: setools-libs-python-3.3.7-4.el6.x86_64. >2014-09-11 10:10:27,629 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 77/113: radvd-1.6-1.el6.x86_64 >2014-09-11 10:10:27,640 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 77/113: radvd-1.6-1.el6.x86_64. >2014-09-11 10:10:27,747 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 78/113: libasyncns-0.8-1.1.el6.x86_64 >2014-09-11 10:10:27,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 78/113: libasyncns-0.8-1.1.el6.x86_64. >2014-09-11 10:10:27,872 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 79/113: pulseaudio-libs-0.9.21-14.el6_3.x86_64 >2014-09-11 10:10:27,882 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 79/113: pulseaudio-libs-0.9.21-14.el6_3.x86_64. >2014-09-11 10:10:28,347 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 80/113: kpartx-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:10:28,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 80/113: kpartx-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:10:28,411 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 81/113: febootstrap-supermin-helper-3.21-4.el6.x86_64 >2014-09-11 10:10:28,420 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 81/113: febootstrap-supermin-helper-3.21-4.el6.x86_64. >2014-09-11 10:10:28,505 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 82/113: python-decorator-3.0.1-3.1.el6.noarch >2014-09-11 10:10:28,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 82/113: python-decorator-3.0.1-3.1.el6.noarch. >2014-09-11 10:10:28,582 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 83/113: pyparted-3.4-4.el6.x86_64 >2014-09-11 10:10:28,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 83/113: pyparted-3.4-4.el6.x86_64. >2014-09-11 10:10:28,775 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 84/113: libsemanage-python-2.0.43-4.2.el6.x86_64 >2014-09-11 10:10:28,791 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 84/113: libsemanage-python-2.0.43-4.2.el6.x86_64. >2014-09-11 10:10:28,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 85/113: policycoreutils-python-2.0.83-19.39.el6.x86_64 >2014-09-11 10:10:28,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 85/113: policycoreutils-python-2.0.83-19.39.el6.x86_64. >2014-09-11 10:11:02,761 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 86/113: fuse-2.8.3-4.el6.x86_64 >2014-09-11 10:11:02,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 86/113: fuse-2.8.3-4.el6.x86_64. >2014-09-11 10:11:02,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 87/113: python-pthreading-0.1.3-3.el6ev.noarch >2014-09-11 10:11:02,869 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 87/113: python-pthreading-0.1.3-3.el6ev.noarch. >2014-09-11 10:11:02,943 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 88/113: nc-1.84-22.el6.x86_64 >2014-09-11 10:11:02,959 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 88/113: nc-1.84-22.el6.x86_64. >2014-09-11 10:11:03,058 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 89/113: libvirt-client-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:11:03,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 89/113: libvirt-client-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:11:03,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 90/113: libvirt-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:11:03,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 90/113: libvirt-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:11:04,749 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 91/113: libvirt-python-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:11:04,767 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 91/113: libvirt-python-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:11:05,094 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 92/113: mom-0.4.1-2.el6.noarch >2014-09-11 10:11:05,106 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 92/113: mom-0.4.1-2.el6.noarch. >2014-09-11 10:11:05,389 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 93/113: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 >2014-09-11 10:11:05,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 93/113: libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64. >2014-09-11 10:11:05,488 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 94/113: fence-agents-3.1.5-35.el6_5.4.x86_64 >2014-09-11 10:11:05,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 94/113: fence-agents-3.1.5-35.el6_5.4.x86_64. >2014-09-11 10:11:05,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 95/113: netpbm-10.47.05-11.el6.x86_64 >2014-09-11 10:11:05,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 95/113: netpbm-10.47.05-11.el6.x86_64. >2014-09-11 10:11:06,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 96/113: netpbm-progs-10.47.05-11.el6.x86_64 >2014-09-11 10:11:06,292 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 96/113: netpbm-progs-10.47.05-11.el6.x86_64. >2014-09-11 10:11:07,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 97/113: gpxe-roms-qemu-0.9.7-6.10.el6.noarch >2014-09-11 10:11:07,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 97/113: gpxe-roms-qemu-0.9.7-6.10.el6.noarch. >2014-09-11 10:11:07,329 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 98/113: sgabios-bin-0-0.3.20110621svn.el6.noarch >2014-09-11 10:11:07,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 98/113: sgabios-bin-0-0.3.20110621svn.el6.noarch. >2014-09-11 10:11:07,381 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 99/113: tree-1.5.3-2.el6.x86_64 >2014-09-11 10:11:07,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 99/113: tree-1.5.3-2.el6.x86_64. >2014-09-11 10:11:07,439 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 100/113: vgabios-0.6b-3.7.el6.noarch >2014-09-11 10:11:07,450 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 100/113: vgabios-0.6b-3.7.el6.noarch. >2014-09-11 10:11:07,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 101/113: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:11:07,538 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 101/113: 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:11:08,800 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 102/113: 1:libguestfs-1.20.11-2.el6.x86_64 >2014-09-11 10:11:08,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 102/113: 1:libguestfs-1.20.11-2.el6.x86_64. >2014-09-11 10:11:09,195 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 103/113: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:11:09,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 103/113: device-mapper-multipath-libs-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:11:09,704 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 104/113: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64 >2014-09-11 10:11:09,717 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 104/113: device-mapper-multipath-0.4.9-72.el6_5.3.x86_64. >2014-09-11 10:11:09,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 105/113: libconfig-1.3.2-1.1.el6.x86_64 >2014-09-11 10:11:09,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 105/113: libconfig-1.3.2-1.1.el6.x86_64. >2014-09-11 10:11:10,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 106/113: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64 >2014-09-11 10:11:10,019 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 106/113: 1:libguestfs-tools-c-1.20.11-2.el6.x86_64. >2014-09-11 10:11:10,549 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 107/113: vdsm-4.16.3-3.el6ev.beta.x86_64 >2014-09-11 10:11:10,565 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 107/113: vdsm-4.16.3-3.el6ev.beta.x86_64. >2014-09-11 10:11:43,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum update: 108/113: audit-2.2-4.el6_5.x86_64 >2014-09-11 10:11:43,988 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum update: 108/113: audit-2.2-4.el6_5.x86_64. >2014-09-11 10:11:44,334 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum install: 109/113: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64 >2014-09-11 10:11:44,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum install: 109/113: 2:qemu-kvm-rhev-tools-0.12.1.2-2.415.el6_5.14.x86_64. >2014-09-11 10:11:44,428 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 110/113: audit >2014-09-11 10:11:44,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 110/113: audit. >2014-09-11 10:11:45,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 111/113: audit-libs >2014-09-11 10:11:45,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 111/113: audit-libs. >2014-09-11 10:11:45,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 112/113: kpartx >2014-09-11 10:11:45,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 112/113: kpartx. >2014-09-11 10:11:45,218 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum updated: 113/113: gnutls >2014-09-11 10:11:45,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum updated: 113/113: gnutls. >2014-09-11 10:11:53,222 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 1/113: libconfig.x86_64 0:1.3.2-1.1.el6 - u >2014-09-11 10:11:53,234 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 1/113: libconfig.x86_64 0:1.3.2-1.1.el6 - u. >2014-09-11 10:11:53,252 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 2/113: device-mapper-multipath-libs.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-11 10:11:53,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 2/113: device-mapper-multipath-libs.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-11 10:11:53,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 3/113: vgabios.noarch 0:0.6b-3.7.el6 - u >2014-09-11 10:11:53,279 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 3/113: vgabios.noarch 0:0.6b-3.7.el6 - u. >2014-09-11 10:11:53,297 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 4/113: tree.x86_64 0:1.5.3-2.el6 - u >2014-09-11 10:11:53,306 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 4/113: tree.x86_64 0:1.5.3-2.el6 - u. >2014-09-11 10:11:53,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 5/113: sgabios-bin.noarch 0:0-0.3.20110621svn.el6 - u >2014-09-11 10:11:53,353 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 5/113: sgabios-bin.noarch 0:0-0.3.20110621svn.el6 - u. >2014-09-11 10:11:53,359 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 6/113: qemu-img-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-11 10:11:53,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 6/113: qemu-img-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-11 10:11:53,372 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 7/113: celt051.x86_64 0:0.5.1.3-0.el6 - u >2014-09-11 10:11:53,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 7/113: celt051.x86_64 0:0.5.1.3-0.el6 - u. >2014-09-11 10:11:53,386 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 8/113: sanlock.x86_64 0:2.8-1.el6 - u >2014-09-11 10:11:53,396 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 8/113: sanlock.x86_64 0:2.8-1.el6 - u. >2014-09-11 10:11:53,401 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 9/113: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-11 10:11:53,409 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 9/113: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-11 10:11:53,419 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 10/113: ioprocess.x86_64 0:0.12.0-2.el6ev - u >2014-09-11 10:11:53,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 10/113: ioprocess.x86_64 0:0.12.0-2.el6ev - u. >2014-09-11 10:11:53,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 11/113: hivex.x86_64 0:1.3.3-4.2.el6 - u >2014-09-11 10:11:53,529 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 11/113: hivex.x86_64 0:1.3.3-4.2.el6 - u. >2014-09-11 10:11:53,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 12/113: vdsm.x86_64 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:53,570 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 12/113: vdsm.x86_64 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:53,576 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 13/113: gettext.x86_64 0:0.17-16.el6 - u >2014-09-11 10:11:53,585 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 13/113: gettext.x86_64 0:0.17-16.el6 - u. >2014-09-11 10:11:53,587 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 14/113: vdsm-python.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:53,596 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 14/113: vdsm-python.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:53,601 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 15/113: gpxe-roms-qemu.noarch 0:0.9.7-6.10.el6 - u >2014-09-11 10:11:53,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 15/113: gpxe-roms-qemu.noarch 0:0.9.7-6.10.el6 - u. >2014-09-11 10:11:53,616 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 16/113: vdsm-cli.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:53,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 16/113: vdsm-cli.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:53,629 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 17/113: netpbm.x86_64 0:10.47.05-11.el6 - u >2014-09-11 10:11:53,646 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 17/113: netpbm.x86_64 0:10.47.05-11.el6 - u. >2014-09-11 10:11:53,647 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 18/113: gnutls.x86_64 0:2.8.5-14.el6_5 - u >2014-09-11 10:11:53,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 18/113: gnutls.x86_64 0:2.8.5-14.el6_5 - u. >2014-09-11 10:11:53,662 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 19/113: nc.x86_64 0:1.84-22.el6 - u >2014-09-11 10:11:53,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 19/113: nc.x86_64 0:1.84-22.el6 - u. >2014-09-11 10:11:53,676 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 20/113: mom.noarch 0:0.4.1-2.el6 - u >2014-09-11 10:11:53,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 20/113: mom.noarch 0:0.4.1-2.el6 - u. >2014-09-11 10:11:53,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 21/113: pyparted.x86_64 0:3.4-4.el6 - u >2014-09-11 10:11:53,702 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 21/113: pyparted.x86_64 0:3.4-4.el6 - u. >2014-09-11 10:11:53,741 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 22/113: python-pthreading.noarch 0:0.1.3-3.el6ev - u >2014-09-11 10:11:53,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 22/113: python-pthreading.noarch 0:0.1.3-3.el6ev - u. >2014-09-11 10:11:53,758 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 23/113: libtirpc.x86_64 0:0.2.1-6.el6_5.2 - u >2014-09-11 10:11:53,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 23/113: libtirpc.x86_64 0:0.2.1-6.el6_5.2 - u. >2014-09-11 10:11:53,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 24/113: flac.x86_64 0:1.2.1-6.1.el6 - u >2014-09-11 10:11:53,781 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 24/113: flac.x86_64 0:1.2.1-6.1.el6 - u. >2014-09-11 10:11:53,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 25/113: fuse.x86_64 0:2.8.3-4.el6 - u >2014-09-11 10:11:53,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 25/113: fuse.x86_64 0:2.8.3-4.el6 - u. >2014-09-11 10:11:53,793 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 26/113: gnutls-utils.x86_64 0:2.8.5-14.el6_5 - u >2014-09-11 10:11:53,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 26/113: gnutls-utils.x86_64 0:2.8.5-14.el6_5 - u. >2014-09-11 10:11:53,806 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 27/113: libsemanage-python.x86_64 0:2.0.43-4.2.el6 - u >2014-09-11 10:11:53,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 27/113: libsemanage-python.x86_64 0:2.0.43-4.2.el6 - u. >2014-09-11 10:11:53,817 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 28/113: libvirt.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-11 10:11:53,828 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 28/113: libvirt.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-11 10:11:53,828 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 29/113: python-decorator.noarch 0:3.0.1-3.1.el6 - u >2014-09-11 10:11:53,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 29/113: python-decorator.noarch 0:3.0.1-3.1.el6 - u. >2014-09-11 10:11:53,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 30/113: febootstrap-supermin-helper.x86_64 0:3.21-4.el6 - u >2014-09-11 10:11:53,850 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 30/113: febootstrap-supermin-helper.x86_64 0:3.21-4.el6 - u. >2014-09-11 10:11:53,854 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 31/113: kpartx.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-11 10:11:53,863 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 31/113: kpartx.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-11 10:11:53,916 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 32/113: libasyncns.x86_64 0:0.8-1.1.el6 - u >2014-09-11 10:11:53,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 32/113: libasyncns.x86_64 0:0.8-1.1.el6 - u. >2014-09-11 10:11:53,949 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 33/113: vdsm-xmlrpc.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:53,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 33/113: vdsm-xmlrpc.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:53,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 34/113: vdsm-python-zombiereaper.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:53,969 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 34/113: vdsm-python-zombiereaper.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:53,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 35/113: radvd.x86_64 0:1.6-1.el6 - u >2014-09-11 10:11:53,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 35/113: radvd.x86_64 0:1.6-1.el6 - u. >2014-09-11 10:11:54,036 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 36/113: usbredir.x86_64 0:0.5.1-1.el6 - u >2014-09-11 10:11:54,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 36/113: usbredir.x86_64 0:0.5.1-1.el6 - u. >2014-09-11 10:11:54,052 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 37/113: setools-libs.x86_64 0:3.3.7-4.el6 - u >2014-09-11 10:11:54,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 37/113: setools-libs.x86_64 0:3.3.7-4.el6 - u. >2014-09-11 10:11:54,065 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 38/113: scrub.x86_64 0:2.2-1.el6 - u >2014-09-11 10:11:54,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 38/113: scrub.x86_64 0:2.2-1.el6 - u. >2014-09-11 10:11:54,080 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 39/113: genisoimage.x86_64 0:1.1.9-12.el6 - u >2014-09-11 10:11:54,091 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 39/113: genisoimage.x86_64 0:1.1.9-12.el6 - u. >2014-09-11 10:11:54,093 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 40/113: libvirt-python.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-11 10:11:54,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 40/113: libvirt-python.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-11 10:11:54,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 41/113: btrfs-progs.x86_64 0:0.20-0.2.git91d9eec.el6 - u >2014-09-11 10:11:54,120 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 41/113: btrfs-progs.x86_64 0:0.20-0.2.git91d9eec.el6 - u. >2014-09-11 10:11:54,121 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 42/113: audit.x86_64 0:2.2-4.el6_5 - u >2014-09-11 10:11:54,132 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 42/113: audit.x86_64 0:2.2-4.el6_5 - u. >2014-09-11 10:11:54,135 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 43/113: vdsm-yajsonrpc.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:54,146 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 43/113: vdsm-yajsonrpc.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:54,156 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 44/113: net-snmp-utils.x86_64 1:5.5-49.el6_5.2 - u >2014-09-11 10:11:54,175 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 44/113: net-snmp-utils.x86_64 1:5.5-49.el6_5.2 - u. >2014-09-11 10:11:54,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 45/113: rpcbind.x86_64 0:0.2.0-11.el6 - u >2014-09-11 10:11:54,188 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 45/113: rpcbind.x86_64 0:0.2.0-11.el6 - u. >2014-09-11 10:11:54,188 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 46/113: python-cpopen.x86_64 0:1.3-4.el6_5 - u >2014-09-11 10:11:54,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 46/113: python-cpopen.x86_64 0:1.3-4.el6_5 - u. >2014-09-11 10:11:54,199 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 47/113: qemu-kvm-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u >2014-09-11 10:11:54,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 47/113: qemu-kvm-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 - u. >2014-09-11 10:11:54,209 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 48/113: numad.x86_64 0:0.5-10.20140620git.el6_5 - u >2014-09-11 10:11:54,220 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 48/113: numad.x86_64 0:0.5-10.20140620git.el6_5 - u. >2014-09-11 10:11:54,220 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 49/113: ghostscript-fonts.noarch 0:5.50-23.1.el6 - u >2014-09-11 10:11:54,230 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 49/113: ghostscript-fonts.noarch 0:5.50-23.1.el6 - u. >2014-09-11 10:11:54,233 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 50/113: keyutils.x86_64 0:1.4-4.el6 - u >2014-09-11 10:11:54,247 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 50/113: keyutils.x86_64 0:1.4-4.el6 - u. >2014-09-11 10:11:54,247 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 51/113: augeas.x86_64 0:1.0.0-5.el6_5.1 - u >2014-09-11 10:11:54,258 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 51/113: augeas.x86_64 0:1.0.0-5.el6_5.1 - u. >2014-09-11 10:11:54,259 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 52/113: pexpect.noarch 0:2.3-6.el6 - u >2014-09-11 10:11:54,269 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 52/113: pexpect.noarch 0:2.3-6.el6 - u. >2014-09-11 10:11:54,269 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 53/113: ebtables.x86_64 0:2.0.9-6.el6 - u >2014-09-11 10:11:54,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 53/113: ebtables.x86_64 0:2.0.9-6.el6 - u. >2014-09-11 10:11:54,290 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 54/113: ghostscript.x86_64 0:8.70-19.el6 - u >2014-09-11 10:11:54,301 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 54/113: ghostscript.x86_64 0:8.70-19.el6 - u. >2014-09-11 10:11:54,301 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 55/113: libguestfs-tools-c.x86_64 1:1.20.11-2.el6 - u >2014-09-11 10:11:54,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 55/113: libguestfs-tools-c.x86_64 1:1.20.11-2.el6 - u. >2014-09-11 10:11:54,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 56/113: nfs-utils-lib.x86_64 0:1.1.5-6.el6_5 - u >2014-09-11 10:11:54,329 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 56/113: nfs-utils-lib.x86_64 0:1.1.5-6.el6_5 - u. >2014-09-11 10:11:54,330 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 57/113: libXfont.x86_64 0:1.4.5-3.el6_5 - u >2014-09-11 10:11:54,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 57/113: libXfont.x86_64 0:1.4.5-3.el6_5 - u. >2014-09-11 10:11:54,347 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 58/113: vdsm-jsonrpc.noarch 0:4.16.3-3.el6ev.beta - u >2014-09-11 10:11:54,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 58/113: vdsm-jsonrpc.noarch 0:4.16.3-3.el6ev.beta - u. >2014-09-11 10:11:54,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 59/113: setools-libs-python.x86_64 0:3.3.7-4.el6 - u >2014-09-11 10:11:54,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 59/113: setools-libs-python.x86_64 0:3.3.7-4.el6 - u. >2014-09-11 10:11:54,397 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 60/113: python-ioprocess.noarch 0:0.12.0-2.el6ev - u >2014-09-11 10:11:54,414 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 60/113: python-ioprocess.noarch 0:0.12.0-2.el6ev - u. >2014-09-11 10:11:54,415 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 61/113: fence-agents.x86_64 0:3.1.5-35.el6_5.4 - u >2014-09-11 10:11:54,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 61/113: fence-agents.x86_64 0:3.1.5-35.el6_5.4 - u. >2014-09-11 10:11:54,427 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 62/113: python-inotify.noarch 0:0.9.1-1.el6ev - u >2014-09-11 10:11:54,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 62/113: python-inotify.noarch 0:0.9.1-1.el6ev - u. >2014-09-11 10:11:54,443 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 63/113: libevent.x86_64 0:1.4.13-4.el6 - u >2014-09-11 10:11:54,458 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 63/113: libevent.x86_64 0:1.4.13-4.el6 - u. >2014-09-11 10:11:54,458 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 64/113: urw-fonts.noarch 0:2.4-10.el6 - u >2014-09-11 10:11:54,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 64/113: urw-fonts.noarch 0:2.4-10.el6 - u. >2014-09-11 10:11:54,469 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 65/113: telnet.x86_64 1:0.17-47.el6_3.1 - u >2014-09-11 10:11:54,480 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 65/113: telnet.x86_64 1:0.17-47.el6_3.1 - u. >2014-09-11 10:11:54,481 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 66/113: audit-libs-python.x86_64 0:2.2-4.el6_5 - u >2014-09-11 10:11:54,493 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 66/113: audit-libs-python.x86_64 0:2.2-4.el6_5 - u. >2014-09-11 10:11:54,494 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 67/113: netcf-libs.x86_64 0:0.1.9-4.el6_5.2 - u >2014-09-11 10:11:54,505 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 67/113: netcf-libs.x86_64 0:0.1.9-4.el6_5.2 - u. >2014-09-11 10:11:54,505 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 68/113: python-suds.noarch 0:0.4.1-3.el6 - u >2014-09-11 10:11:54,521 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 68/113: python-suds.noarch 0:0.4.1-3.el6 - u. >2014-09-11 10:11:54,521 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 69/113: libcgroup.x86_64 0:0.40.rc1-5.el6_5.1 - u >2014-09-11 10:11:54,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 69/113: libcgroup.x86_64 0:0.40.rc1-5.el6_5.1 - u. >2014-09-11 10:11:54,533 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 70/113: seabios.x86_64 0:0.6.1.2-28.el6 - u >2014-09-11 10:11:54,546 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 70/113: seabios.x86_64 0:0.6.1.2-28.el6 - u. >2014-09-11 10:11:54,546 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 71/113: libogg.x86_64 2:1.1.4-2.1.el6 - u >2014-09-11 10:11:54,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 71/113: libogg.x86_64 2:1.1.4-2.1.el6 - u. >2014-09-11 10:11:54,558 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 72/113: libXtst.x86_64 0:1.2.1-2.el6 - u >2014-09-11 10:11:54,570 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 72/113: libXtst.x86_64 0:1.2.1-2.el6 - u. >2014-09-11 10:11:54,570 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 73/113: fuse-libs.x86_64 0:2.8.3-4.el6 - u >2014-09-11 10:11:54,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 73/113: fuse-libs.x86_64 0:2.8.3-4.el6 - u. >2014-09-11 10:11:54,582 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 74/113: libsndfile.x86_64 0:1.0.20-5.el6 - u >2014-09-11 10:11:54,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 74/113: libsndfile.x86_64 0:1.0.20-5.el6 - u. >2014-09-11 10:11:54,594 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 75/113: perl-Net-Telnet.noarch 0:3.03-11.el6 - u >2014-09-11 10:11:54,606 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 75/113: perl-Net-Telnet.noarch 0:3.03-11.el6 - u. >2014-09-11 10:11:54,607 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 76/113: net-snmp-libs.x86_64 1:5.5-49.el6_5.2 - u >2014-09-11 10:11:54,616 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 76/113: net-snmp-libs.x86_64 1:5.5-49.el6_5.2 - u. >2014-09-11 10:11:54,616 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 77/113: policycoreutils-python.x86_64 0:2.0.83-19.39.el6 - u >2014-09-11 10:11:54,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 77/113: policycoreutils-python.x86_64 0:2.0.83-19.39.el6 - u. >2014-09-11 10:11:54,625 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 78/113: yajl.x86_64 0:1.0.7-3.el6 - u >2014-09-11 10:11:54,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 78/113: yajl.x86_64 0:1.0.7-3.el6 - u. >2014-09-11 10:11:54,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 79/113: cvs.x86_64 0:1.11.23-16.el6 - u >2014-09-11 10:11:54,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 79/113: cvs.x86_64 0:1.11.23-16.el6 - u. >2014-09-11 10:11:54,648 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 80/113: nfs-utils.x86_64 1:1.2.3-39.el6_5.3 - u >2014-09-11 10:11:54,657 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 80/113: nfs-utils.x86_64 1:1.2.3-39.el6_5.3 - u. >2014-09-11 10:11:54,657 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 81/113: audit-libs.x86_64 0:2.2-4.el6_5 - u >2014-09-11 10:11:54,665 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 81/113: audit-libs.x86_64 0:2.2-4.el6_5 - u. >2014-09-11 10:11:54,665 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 82/113: sanlock-lib.x86_64 0:2.8-1.el6 - u >2014-09-11 10:11:54,676 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 82/113: sanlock-lib.x86_64 0:2.8-1.el6 - u. >2014-09-11 10:11:54,676 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 83/113: netpbm-progs.x86_64 0:10.47.05-11.el6 - u >2014-09-11 10:11:54,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 83/113: netpbm-progs.x86_64 0:10.47.05-11.el6 - u. >2014-09-11 10:11:54,686 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 84/113: python-argparse.noarch 0:1.2.1-2.1.el6 - u >2014-09-11 10:11:54,695 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 84/113: python-argparse.noarch 0:1.2.1-2.1.el6 - u. >2014-09-11 10:11:54,695 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 85/113: libvorbis.x86_64 1:1.2.3-4.el6_2.1 - u >2014-09-11 10:11:54,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 85/113: libvorbis.x86_64 1:1.2.3-4.el6_2.1 - u. >2014-09-11 10:11:54,703 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 86/113: sg3_utils.x86_64 0:1.28-5.el6 - u >2014-09-11 10:11:54,712 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 86/113: sg3_utils.x86_64 0:1.28-5.el6 - u. >2014-09-11 10:11:54,712 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 87/113: spice-server.x86_64 0:0.12.4-6.el6_5.2 - u >2014-09-11 10:11:54,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 87/113: spice-server.x86_64 0:0.12.4-6.el6_5.2 - u. >2014-09-11 10:11:54,721 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 88/113: glusterfs.x86_64 0:3.5.2-1.el6 - u >2014-09-11 10:11:54,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 88/113: glusterfs.x86_64 0:3.5.2-1.el6 - u. >2014-09-11 10:11:54,732 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 89/113: perl-hivex.x86_64 0:1.3.3-4.2.el6 - u >2014-09-11 10:11:54,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 89/113: perl-hivex.x86_64 0:1.3.3-4.2.el6 - u. >2014-09-11 10:11:54,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 90/113: pulseaudio-libs.x86_64 0:0.9.21-14.el6_3 - u >2014-09-11 10:11:54,753 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 90/113: pulseaudio-libs.x86_64 0:0.9.21-14.el6_3 - u. >2014-09-11 10:11:54,759 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 91/113: python-ordereddict.noarch 0:1.1-3.el6ev - u >2014-09-11 10:11:54,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 91/113: python-ordereddict.noarch 0:1.1-3.el6ev - u. >2014-09-11 10:11:54,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 92/113: libfontenc.x86_64 0:1.0.5-2.el6 - u >2014-09-11 10:11:54,779 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 92/113: libfontenc.x86_64 0:1.0.5-2.el6 - u. >2014-09-11 10:11:54,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 93/113: xorg-x11-font-utils.x86_64 1:7.2-11.el6 - u >2014-09-11 10:11:54,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 93/113: xorg-x11-font-utils.x86_64 1:7.2-11.el6 - u. >2014-09-11 10:11:54,792 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 94/113: lzop.x86_64 0:1.02-0.9.rc1.el6 - u >2014-09-11 10:11:54,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 94/113: lzop.x86_64 0:1.02-0.9.rc1.el6 - u. >2014-09-11 10:11:54,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 95/113: libvirt-client.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-11 10:11:54,814 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 95/113: libvirt-client.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-11 10:11:54,819 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 96/113: sanlock-python.x86_64 0:2.8-1.el6 - u >2014-09-11 10:11:54,828 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 96/113: sanlock-python.x86_64 0:2.8-1.el6 - u. >2014-09-11 10:11:54,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 97/113: glusterfs-api.x86_64 0:3.5.2-1.el6 - u >2014-09-11 10:11:54,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 97/113: glusterfs-api.x86_64 0:3.5.2-1.el6 - u. >2014-09-11 10:11:54,845 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 98/113: dnsmasq.x86_64 0:2.48-13.el6 - u >2014-09-11 10:11:54,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 98/113: dnsmasq.x86_64 0:2.48-13.el6 - u. >2014-09-11 10:11:54,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 99/113: python-netaddr.noarch 0:0.7.5-4.el6 - u >2014-09-11 10:11:54,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 99/113: python-netaddr.noarch 0:0.7.5-4.el6 - u. >2014-09-11 10:11:54,891 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 100/113: libgomp.x86_64 0:4.4.7-4.el6 - u >2014-09-11 10:11:54,900 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 100/113: libgomp.x86_64 0:4.4.7-4.el6 - u. >2014-09-11 10:11:54,908 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 101/113: device-mapper-multipath.x86_64 0:0.4.9-72.el6_5.3 - u >2014-09-11 10:11:54,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 101/113: device-mapper-multipath.x86_64 0:0.4.9-72.el6_5.3 - u. >2014-09-11 10:11:54,921 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 102/113: ipmitool.x86_64 0:1.8.11-16.el6 - u >2014-09-11 10:11:54,930 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 102/113: ipmitool.x86_64 0:1.8.11-16.el6 - u. >2014-09-11 10:11:54,932 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 103/113: cyrus-sasl-md5.x86_64 0:2.1.23-13.el6_3.1 - u >2014-09-11 10:11:54,940 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 103/113: cyrus-sasl-md5.x86_64 0:2.1.23-13.el6_3.1 - u. >2014-09-11 10:11:54,949 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 104/113: augeas-libs.x86_64 0:1.0.0-5.el6_5.1 - u >2014-09-11 10:11:54,964 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 104/113: augeas-libs.x86_64 0:1.0.0-5.el6_5.1 - u. >2014-09-11 10:11:54,964 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 105/113: lm_sensors-libs.x86_64 0:3.1.1-17.el6 - u >2014-09-11 10:11:54,972 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 105/113: lm_sensors-libs.x86_64 0:3.1.1-17.el6 - u. >2014-09-11 10:11:54,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 106/113: libguestfs.x86_64 1:1.20.11-2.el6 - u >2014-09-11 10:11:54,985 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 106/113: libguestfs.x86_64 1:1.20.11-2.el6 - u. >2014-09-11 10:11:54,990 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 107/113: glusterfs-libs.x86_64 0:3.5.2-1.el6 - u >2014-09-11 10:11:54,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 107/113: glusterfs-libs.x86_64 0:3.5.2-1.el6 - u. >2014-09-11 10:11:55,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 108/113: libgssglue.x86_64 0:0.1-11.el6 - u >2014-09-11 10:11:55,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 108/113: libgssglue.x86_64 0:0.1-11.el6 - u. >2014-09-11 10:11:55,014 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 109/113: libvirt-lock-sanlock.x86_64 0:0.10.2-29.el6_5.12 - u >2014-09-11 10:11:55,022 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 109/113: libvirt-lock-sanlock.x86_64 0:0.10.2-29.el6_5.12 - u. >2014-09-11 10:11:55,026 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 110/113: kpartx.x86_64 0:0.4.9-72.el6 - ud >2014-09-11 10:11:55,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 110/113: kpartx.x86_64 0:0.4.9-72.el6 - ud. >2014-09-11 10:11:55,038 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 111/113: audit.x86_64 0:2.2-2.el6 - ud >2014-09-11 10:11:55,052 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 111/113: audit.x86_64 0:2.2-2.el6 - ud. >2014-09-11 10:11:55,052 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 112/113: audit-libs.x86_64 0:2.2-2.el6 - ud >2014-09-11 10:11:55,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 112/113: audit-libs.x86_64 0:2.2-2.el6 - ud. >2014-09-11 10:11:55,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Yum Verify: 113/113: gnutls.x86_64 0:2.8.5-10.el6_4.2 - ud >2014-09-11 10:11:55,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Yum Verify: 113/113: gnutls.x86_64 0:2.8.5-10.el6_4.2 - ud. >2014-09-11 10:11:55,122 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Misc configuration >2014-09-11 10:11:55,130 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Misc configuration. >2014-09-11 10:11:57,223 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Enrolling certificate >2014-09-11 10:11:57,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Enrolling certificate. >2014-09-11 10:11:58,381 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Transaction commit >2014-09-11 10:11:58,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Transaction commit. >2014-09-11 10:11:58,830 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Closing up >2014-09-11 10:11:58,838 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Closing up. >2014-09-11 10:12:00,534 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stopping libvirtd >2014-09-11 10:12:00,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stopping libvirtd. >2014-09-11 10:12:00,629 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Starting vdsm >2014-09-11 10:12:00,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Starting vdsm. >2014-09-11 10:12:06,787 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Pre-termination >2014-09-11 10:12:06,796 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Pre-termination. >2014-09-11 10:12:06,860 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140911101206-10.34.66.17-6bbde29d.log' >2014-09-11 10:12:06,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140911101206-10.34.66.17-6bbde29d.log'. >2014-09-11 10:12:07,952 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation 10.34.66.17: Stage: Termination >2014-09-11 10:12:07,961 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 6bbde29d, Call Stack: null, Custom Event ID: -1, Message: Installing Host 10.34.66.17. Stage: Termination. >2014-09-11 10:12:08,748 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.34.66.17 >2014-09-11 10:12:08,856 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-20) [6bbde29d] Engine managed to communicate with VDSM agent on host 10.34.66.17 >2014-09-11 10:12:09,349 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-20) [681b786e] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-11 10:12:09,355 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-20) [681b786e] START, SetupNetworksVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[eth0 {id=29982636-6cee-435f-87ba-9e626cc22c70, vdsId=73e23c28-b882-49ee-946e-b4837bf1ec9f, macAddress=00:1a:4a:51:b2:04, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.66.17, subnet=255.255.255.0, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 1975de71 >2014-09-11 10:12:09,359 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-20) [681b786e] FINISH, SetupNetworksVDSCommand, log id: 1975de71 >2014-09-11 10:12:09,361 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.34.66.17 >2014-09-11 10:12:18,194 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.34.66.17 >2014-09-11 10:12:18,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) Correlation ID: 681b786e, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-11 10:12:18,497 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-11 10:12:18,500 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] START, SetSafeNetworkConfigVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: f8356a4 >2014-09-11 10:12:18,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] FINISH, SetSafeNetworkConfigVDSCommand, log id: f8356a4 >2014-09-11 10:12:18,547 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [4213a58] Correlation ID: 4213a58, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host 10.34.66.17 >2014-09-11 10:12:18,550 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 191909be >2014-09-11 10:12:18,567 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] FINISH, SetVdsStatusVDSCommand, log id: 191909be >2014-09-11 10:12:18,568 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] After Installation host 10.34.66.17, VDS >2014-09-11 10:12:18,593 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [4213a58] Correlation ID: 6bbde29d, Job ID: 0f4fd529-b94d-4678-9f4a-fd207177216c, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 installed >2014-09-11 10:12:18,601 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [4213a58] Lock freed to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:12:21,343 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) START, GetHardwareInfoVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, vds=Host[10.34.66.17,73e23c28-b882-49ee-946e-b4837bf1ec9f]), log id: 283d6e2b >2014-09-11 10:12:21,621 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, GetHardwareInfoVDSCommand, log id: 283d6e2b >2014-09-11 10:12:21,663 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-12) [4002f8a3] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:21,720 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:21,724 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 1b9698d1 >2014-09-11 10:12:21,748 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] FINISH, SetVdsStatusVDSCommand, log id: 1b9698d1 >2014-09-11 10:12:21,798 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] Correlation ID: 6ef2d8c8, Job ID: f752ae42-d2f6-4823-9ae5-4359d4faaafc, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 moved to Non-Operational state. >2014-09-11 10:12:21,821 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] Correlation ID: 4002f8a3, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-11 10:12:21,890 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] START, GetHardwareInfoVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, vds=Host[10.34.66.17,73e23c28-b882-49ee-946e-b4837bf1ec9f]), log id: 54a3fa >2014-09-11 10:12:21,897 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-12) [6ef2d8c8] FINISH, GetHardwareInfoVDSCommand, log id: 54a3fa >2014-09-11 10:12:21,933 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-12) [6580425c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:21,975 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-12) [41997104] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:21,978 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [41997104] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 6c52f5b3 >2014-09-11 10:12:21,989 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [41997104] FINISH, SetVdsStatusVDSCommand, log id: 6c52f5b3 >2014-09-11 10:12:22,015 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) [41997104] Correlation ID: 6580425c, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-11 10:12:22,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) [41997104] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host 10.34.66.17 was set to NonOperational. >2014-09-11 10:12:22,071 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-12) [249ab029] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:22,148 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-12) [18c137be] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:22,151 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [18c137be] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 697f6f4a >2014-09-11 10:12:22,173 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) [18c137be] FINISH, SetVdsStatusVDSCommand, log id: 697f6f4a >2014-09-11 10:12:22,215 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) [18c137be] Correlation ID: 249ab029, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-11 10:12:22,219 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:12:22,222 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Host 73e23c28-b882-49ee-946e-b4837bf1ec9f : 10.34.66.17 is already in NonOperational status for reason CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-09-11 10:12:57,897 INFO [org.ovirt.engine.core.bll.storage.AddEmptyStoragePoolCommand] (ajp-/127.0.0.1:8702-1) [72441ffb] Running command: AddEmptyStoragePoolCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_POOL with role type ADMIN >2014-09-11 10:12:58,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [72441ffb] Correlation ID: 72441ffb, Call Stack: null, Custom Event ID: -1, Message: Data Center old, Compatibility Version 3.5 and Quota Type DISABLED was added by admin >2014-09-11 10:13:12,384 INFO [org.ovirt.engine.core.bll.AddVdsGroupCommand] (ajp-/127.0.0.1:8702-9) [5804202a] Running command: AddVdsGroupCommand internal: false. Entities affected : ID: cbe33e45-5a46-45be-bb0f-9be614031c2b Type: StoragePoolAction group CREATE_CLUSTER with role type ADMIN >2014-09-11 10:13:12,442 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [5804202a] Correlation ID: 5804202a, Call Stack: null, Custom Event ID: -1, Message: Cluster so_old was added by admin >2014-09-11 10:13:22,085 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-32) [296a14fe] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-11 10:13:22,091 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-32) [296a14fe] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 658d06b2 >2014-09-11 10:13:22,112 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-32) [296a14fe] FINISH, SetVdsStatusVDSCommand, log id: 658d06b2 >2014-09-11 10:13:22,226 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-32) [296a14fe] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:13:22,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-32) [296a14fe] Correlation ID: 296a14fe, Job ID: 7b109ec8-d121-4a7a-92cf-863f20e37a80, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 was switched to Maintenance mode by admin. >2014-09-11 10:13:22,334 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 73e23c28-b882-49ee-946e-b4837bf1ec9f : 10.34.66.17 >2014-09-11 10:13:22,359 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-69) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Available memory of host 10.34.66.17 [745 MB] is under defined threshold [1024 MB]. >2014-09-11 10:13:22,384 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-34) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: 10.34.66.17. >2014-09-11 10:13:22,384 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-34) Removing vds [73e23c28-b882-49ee-946e-b4837bf1ec9f] from the domain in maintenance cache >2014-09-11 10:13:22,385 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-34) Removing host(s) [73e23c28-b882-49ee-946e-b4837bf1ec9f] from hosts unseen domain report cache >2014-09-11 10:13:22,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-69) START, DisconnectStoragePoolVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 2), log id: 465e0f46 >2014-09-11 10:13:22,399 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-69) FINISH, DisconnectStoragePoolVDSCommand, log id: 465e0f46 >2014-09-11 10:13:22,402 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-69) [2589e2b6] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-11 10:13:22,409 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-69) [2589e2b6] START, DisconnectStorageServerVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5e841cbe >2014-09-11 10:13:22,514 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-69) [2589e2b6] FINISH, DisconnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=477}, log id: 5e841cbe >2014-09-11 10:13:43,225 INFO [org.ovirt.engine.core.bll.ChangeVDSClusterCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] Running command: ChangeVDSClusterCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN, ID: 9518e1e3-f526-49a6-8820-7139fc3664f8 Type: VdsGroupsAction group EDIT_HOST_CONFIGURATION with role type ADMIN >2014-09-11 10:13:43,277 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] START, RemoveVdsVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 276134b >2014-09-11 10:13:43,277 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [3f37f5c3] vdsManager::disposing >2014-09-11 10:13:43,289 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] FINISH, RemoveVdsVDSCommand, log id: 276134b >2014-09-11 10:13:43,293 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] START, AddVdsVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 4c3d0ad7 >2014-09-11 10:13:43,293 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] AddVds - entered , starting logic to add VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f >2014-09-11 10:13:43,295 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] AddVds - VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f was added, will try to add it to the resource manager >2014-09-11 10:13:43,296 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [3f37f5c3] Entered VdsManager constructor >2014-09-11 10:13:43,310 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-2) [3f37f5c3] Initialize vdsBroker (10.34.66.17,54,321) >2014-09-11 10:13:43,313 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-2) [3f37f5c3] VDS 73e23c28-b882-49ee-946e-b4837bf1ec9f was added to the Resource Manager >2014-09-11 10:13:43,317 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-2) [3f37f5c3] FINISH, AddVdsVDSCommand, log id: 4c3d0ad7 >2014-09-11 10:13:43,325 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [5dcf60fd] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:13:43,354 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-2) [5dcf60fd] Lock freed to object EngineLock [exclusiveLocks= key: cbe33e45-5a46-45be-bb0f-9be614031c2b value: REGISTER_VDS >, sharedLocks= ] >2014-09-11 10:13:43,398 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [5dcf60fd] Correlation ID: 3f37f5c3, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 configuration was updated by admin. >2014-09-11 10:13:43,401 INFO [org.ovirt.engine.core.bll.network.host.PersistentSetupNetworksCommand] (org.ovirt.thread.pool-6-thread-45) [1f3e4910] Running command: PersistentSetupNetworksCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:13:43,438 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-45) [1f3e4910] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-11 10:13:43,439 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-45) [1f3e4910] No changes were detected in setup networks for host 10.34.66.17 (ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f) >2014-09-11 10:13:43,536 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-11 10:13:43,537 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-11 10:13:43,537 INFO [org.ovirt.engine.core.bll.UpdateVdsCommand] (ajp-/127.0.0.1:8702-4) [51d88c5c] Running command: UpdateVdsCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN >2014-09-11 10:13:43,564 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-11 10:13:43,565 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-4) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-11 10:13:43,592 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [51d88c5c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host 10.34.66.17. >2014-09-11 10:13:43,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [51d88c5c] Correlation ID: 51d88c5c, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 configuration was updated by admin. >2014-09-11 10:13:46,315 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-32) Initializing Host: 10.34.66.17 >2014-09-11 10:15:11,215 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-9) [5e5968da] Lock Acquired to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:15:11,254 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-11 10:15:11,255 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Before acquiring lock in order to prevent monitoring for host 10.34.66.17 from data-center old >2014-09-11 10:15:11,255 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Lock acquired, from now a monitoring of host will be skipped for host 10.34.66.17 from data-center old >2014-09-11 10:15:11,275 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: d5af56 >2014-09-11 10:15:11,300 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] FINISH, SetVdsStatusVDSCommand, log id: d5af56 >2014-09-11 10:15:11,314 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Activate finished. Lock released. Monitoring can run now for host 10.34.66.17 from data-center old >2014-09-11 10:15:11,324 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Correlation ID: 5e5968da, Job ID: 226914ac-eacb-41dd-9102-58822e352a49, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 was activated by admin. >2014-09-11 10:15:11,334 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-36) [5e5968da] Lock freed to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:15:13,440 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to /10.34.66.17 >2014-09-11 10:15:13,643 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GetHardwareInfoVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, vds=Host[10.34.66.17,73e23c28-b882-49ee-946e-b4837bf1ec9f]), log id: 124aeb5b >2014-09-11 10:15:13,675 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GetHardwareInfoVDSCommand, log id: 124aeb5b >2014-09-11 10:15:13,707 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-64) [3cda561d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:15:13,775 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) [3cda561d] START, GetHardwareInfoVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, vds=Host[10.34.66.17,73e23c28-b882-49ee-946e-b4837bf1ec9f]), log id: 35d5fef6 >2014-09-11 10:15:13,780 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-64) [3cda561d] FINISH, GetHardwareInfoVDSCommand, log id: 35d5fef6 >2014-09-11 10:15:13,807 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-64) [7c2f0938] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: cbe33e45-5a46-45be-bb0f-9be614031c2b Type: StoragePool >2014-09-11 10:15:13,826 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-64) [7c2f0938] START, SetMOMPolicyParametersVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 14a9b958 >2014-09-11 10:15:13,898 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-64) [7c2f0938] FINISH, SetMOMPolicyParametersVDSCommand, log id: 14a9b958 >2014-09-11 10:15:13,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) [7c2f0938] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host 10.34.66.17 was set to Up. >2014-09-11 10:15:13,916 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) [7c2f0938] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host 10.34.66.17. >2014-09-11 10:15:13,923 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-64) [7c2f0938] Correlation ID: 7c2f0938, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-11 10:15:13,959 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-64) [435a3a3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:15:17,036 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-70) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Available memory of host 10.34.66.17 [742 MB] is under defined threshold [1024 MB]. >2014-09-11 10:39:46,042 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-49) [5d3c33f3] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-11 10:39:46,045 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d3c33f3] START, SetVdsStatusVDSCommand(HostName = 10.34.66.17, HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 12c50925 >2014-09-11 10:39:46,063 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [5d3c33f3] FINISH, SetVdsStatusVDSCommand, log id: 12c50925 >2014-09-11 10:39:46,209 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-49) [5d3c33f3] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDS >2014-09-11 10:39:46,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-49) [5d3c33f3] Correlation ID: 5d3c33f3, Job ID: c657345e-be55-4b76-9219-66fe18f1ad81, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 was switched to Maintenance mode by admin. >2014-09-11 10:39:47,037 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 73e23c28-b882-49ee-946e-b4837bf1ec9f : 10.34.66.17 >2014-09-11 10:39:55,700 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-2) [4b516b2] Lock Acquired to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:39:55,723 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-7) [4b516b2] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 73e23c28-b882-49ee-946e-b4837bf1ec9f Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-11 10:39:55,823 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-7) [4b516b2] START, RemoveVdsVDSCommand( HostId = 73e23c28-b882-49ee-946e-b4837bf1ec9f), log id: 38433266 >2014-09-11 10:39:55,823 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-7) [4b516b2] vdsManager::disposing >2014-09-11 10:39:55,824 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-7) [4b516b2] FINISH, RemoveVdsVDSCommand, log id: 38433266 >2014-09-11 10:39:55,845 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-7) [4b516b2] Correlation ID: 4b516b2, Call Stack: null, Custom Event ID: -1, Message: Host 10.34.66.17 was removed by admin. >2014-09-11 10:39:55,848 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-7) [4b516b2] Lock freed to object EngineLock [exclusiveLocks= key: 73e23c28-b882-49ee-946e-b4837bf1ec9f value: VDS >, sharedLocks= ] >2014-09-11 10:54:39,616 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-11 10:54:40,318 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) Attempting to update VMs/Templates Ovf. >2014-09-11 10:54:40,629 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 10:54:40,633 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 10:54:40,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 93a339f >2014-09-11 10:54:40,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 10:54:40,711 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 10:54:41,138 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [78c513f9] FINISH, SetVolumeDescriptionVDSCommand, log id: 93a339f >2014-09-11 10:54:41,312 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 10:54:41,327 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 2651646b >2014-09-11 10:54:41,328 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- executeVdsBrokerCommand: >2014-09-11 10:54:41,328 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=10,240 > >2014-09-11 10:54:44,217 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] FINISH, UploadStreamVDSCommand, log id: 2651646b >2014-09-11 10:54:44,360 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-92) [76f846dc] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command f9fa22c5-4418-456e-b02e-b581a4dd2921 >2014-09-11 10:54:44,360 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-92) [76f846dc] CommandMultiAsyncTasks::AttachTask: Attaching task 1d780400-9767-40ee-b234-e9b874bedd33 to command f9fa22c5-4418-456e-b02e-b581a4dd2921. >2014-09-11 10:54:44,401 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) [76f846dc] Adding task 1d780400-9767-40ee-b234-e9b874bedd33 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-11 10:54:44,445 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 10:54:44,462 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 5097b602 >2014-09-11 10:54:44,462 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 10:54:44,462 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 10:54:40 CEST 2014","Size":10240} > >2014-09-11 10:54:44,727 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] FINISH, SetVolumeDescriptionVDSCommand, log id: 5097b602 >2014-09-11 10:54:44,743 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 1dccb0fd >2014-09-11 10:54:44,743 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 10:54:44,744 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":null} > >2014-09-11 10:54:45,061 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [76f846dc] FINISH, SetVolumeDescriptionVDSCommand, log id: 1dccb0fd >2014-09-11 10:54:45,105 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 10:54:45,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 2ac7f70b >2014-09-11 10:54:45,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] -- executeVdsBrokerCommand: >2014-09-11 10:54:45,110 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=10,240 > >2014-09-11 10:54:46,952 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] FINISH, UploadStreamVDSCommand, log id: 2ac7f70b >2014-09-11 10:54:46,956 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-92) [13398f04] CommandMultiAsyncTasks::AttachTask: Attaching task b5d405bf-e537-4a42-8b7d-7cb154412f92 to command f9fa22c5-4418-456e-b02e-b581a4dd2921. >2014-09-11 10:54:46,996 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-92) [13398f04] Adding task b5d405bf-e537-4a42-8b7d-7cb154412f92 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-11 10:54:47,022 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-92) [13398f04] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 10:54:47,023 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-92) [13398f04] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 10:54:47,023 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-92) [13398f04] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 10:54:47,023 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-92) [13398f04] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 10:54:47,023 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 10:54:47,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 68200efa >2014-09-11 10:54:47,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 10:54:47,027 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 10:54:40 CEST 2014","Size":10240} > >2014-09-11 10:54:47,290 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-92) [13398f04] FINISH, SetVolumeDescriptionVDSCommand, log id: 68200efa >2014-09-11 10:54:47,296 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-92) [13398f04] BaseAsyncTask::startPollingTask: Starting to poll task 1d780400-9767-40ee-b234-e9b874bedd33. >2014-09-11 10:54:47,296 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-92) [13398f04] BaseAsyncTask::startPollingTask: Starting to poll task b5d405bf-e537-4a42-8b7d-7cb154412f92. >2014-09-11 10:54:49,580 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-22) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-09-11 10:54:49,593 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task b5d405bf-e537-4a42-8b7d-7cb154412f92 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-11 10:54:49,593 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task b5d405bf-e537-4a42-8b7d-7cb154412f92 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-11 10:54:49,594 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-22) Task with DB Task ID 54f868c7-3a78-41b0-9cd8-99a5b84a52ac and VDSM Task ID 1d780400-9767-40ee-b234-e9b874bedd33 is in state Polling. End action for command f9fa22c5-4418-456e-b02e-b581a4dd2921 will proceed when all the entitys tasks are completed. >2014-09-11 10:54:49,595 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) SPMAsyncTask::PollTask: Polling task 1d780400-9767-40ee-b234-e9b874bedd33 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-11 10:54:49,595 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-22) BaseAsyncTask::onTaskEndSuccess: Task 1d780400-9767-40ee-b234-e9b874bedd33 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-11 10:54:49,595 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endActionIfNecessary: All tasks of command f9fa22c5-4418-456e-b02e-b581a4dd2921 has ended -> executing endAction >2014-09-11 10:54:49,595 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-22) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: f9fa22c5-4418-456e-b02e-b581a4dd2921): calling endAction . >2014-09-11 10:54:49,596 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-31) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-09-11 10:54:49,668 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-09-11 10:54:49,668 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 10:54:49,669 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 10:54:49,669 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 10:54:49,669 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 10:54:49,669 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 10:54:49,670 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-09-11 10:54:49,670 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-09-11 10:54:49,670 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task b5d405bf-e537-4a42-8b7d-7cb154412f92 >2014-09-11 10:54:49,671 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = b5d405bf-e537-4a42-8b7d-7cb154412f92), log id: 71d2d08d >2014-09-11 10:54:49,675 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=b5d405bf-e537-4a42-8b7d-7cb154412f92), log id: 55c4577b >2014-09-11 10:54:49,753 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] FINISH, HSMClearTaskVDSCommand, log id: 55c4577b >2014-09-11 10:54:49,753 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] FINISH, SPMClearTaskVDSCommand, log id: 71d2d08d >2014-09-11 10:54:49,784 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] BaseAsyncTask::removeTaskFromDB: Removed task b5d405bf-e537-4a42-8b7d-7cb154412f92 from DataBase >2014-09-11 10:54:49,784 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 1d780400-9767-40ee-b234-e9b874bedd33 >2014-09-11 10:54:49,785 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 1d780400-9767-40ee-b234-e9b874bedd33), log id: 6762a08c >2014-09-11 10:54:49,787 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=1d780400-9767-40ee-b234-e9b874bedd33), log id: 7c1a7c5a >2014-09-11 10:54:49,869 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] FINISH, HSMClearTaskVDSCommand, log id: 7c1a7c5a >2014-09-11 10:54:49,869 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-31) [78c513f9] FINISH, SPMClearTaskVDSCommand, log id: 6762a08c >2014-09-11 10:54:49,880 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] BaseAsyncTask::removeTaskFromDB: Removed task 1d780400-9767-40ee-b234-e9b874bedd33 from DataBase >2014-09-11 10:54:49,880 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-31) [78c513f9] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity f9fa22c5-4418-456e-b02e-b581a4dd2921 >2014-09-11 10:56:09,284 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) Setting new tasks map. The map contains now 0 tasks >2014-09-11 10:56:09,284 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-96) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-11 11:54:39,635 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-11 11:54:47,298 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-09-11 12:54:39,648 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-66) Principal admin::internal synchronized >2014-09-11 12:54:47,325 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-11 13:54:39,662 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-11 13:54:47,351 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VMs/Templates Ovf. >2014-09-11 14:17:51,048 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-1) [6fe23fb6] Lock Acquired to object EngineLock [exclusiveLocks= key: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export value: STORAGE_CONNECTION >, sharedLocks= ] >2014-09-11 14:17:51,103 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-1) [6fe23fb6] Running command: AddStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:17:51,108 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-1) [6fe23fb6] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: null, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 19452e64 >2014-09-11 14:17:51,411 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-1) [6fe23fb6] FINISH, ConnectStorageServerVDSCommand, return: {00000000-0000-0000-0000-000000000000=0}, log id: 19452e64 >2014-09-11 14:17:51,445 INFO [org.ovirt.engine.core.bll.storage.AddStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-1) [6fe23fb6] Lock freed to object EngineLock [exclusiveLocks= key: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export value: STORAGE_CONNECTION >, sharedLocks= ] >2014-09-11 14:17:57,509 INFO [org.ovirt.engine.core.bll.storage.AddNFSStorageDomainCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] Running command: AddNFSStorageDomainCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:17:57,557 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 58c2081 >2014-09-11 14:17:57,574 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0}, log id: 58c2081 >2014-09-11 14:17:57,584 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] START, CreateStorageDomainVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storageDomain=org.ovirt.engine.core.common.businessentities.StorageDomainStatic@f70d8de, args=10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export), log id: 7d6c3500 >2014-09-11 14:17:59,428 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] FINISH, CreateStorageDomainVDSCommand, log id: 7d6c3500 >2014-09-11 14:17:59,430 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] START, GetStorageDomainStatsVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storageDomainId=3573f707-d0c4-4dc9-b09f-dca9a3506a4f), log id: 1c128296 >2014-09-11 14:17:59,451 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStorageDomainStatsVDSCommand] (ajp-/127.0.0.1:8702-2) [299b1ae] FINISH, GetStorageDomainStatsVDSCommand, return: org.ovirt.engine.core.common.businessentities.StorageDomain@bded2945, log id: 1c128296 >2014-09-11 14:17:59,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [299b1ae] Correlation ID: 299b1ae, Job ID: 44d483f9-64b3-4a1b-97c5-b22875d31cbd, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export was added by admin >2014-09-11 14:18:05,040 INFO [org.ovirt.engine.core.bll.storage.DisconnectStorageServerConnectionCommand] (ajp-/127.0.0.1:8702-4) [64251cbc] Running command: DisconnectStorageServerConnectionCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:18:05,051 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-4) [64251cbc] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5ba61d4c >2014-09-11 14:18:05,204 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (ajp-/127.0.0.1:8702-4) [64251cbc] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0}, log id: 5ba61d4c >2014-09-11 14:18:06,481 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-8) [1d6bad4c] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: 3573f707-d0c4-4dc9-b09f-dca9a3506a4f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:18:06,523 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-3) [15c26e6d] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:18:06,531 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-3) [15c26e6d] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6111f6e3 >2014-09-11 14:18:06,569 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-3) [15c26e6d] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0}, log id: 6111f6e3 >2014-09-11 14:18:06,574 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-8) [1d6bad4c] START, AttachStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = 3573f707-d0c4-4dc9-b09f-dca9a3506a4f), log id: a1454c9 >2014-09-11 14:18:27,913 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-8) [1d6bad4c] FINISH, AttachStorageDomainVDSCommand, log id: a1454c9 >2014-09-11 14:18:27,979 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-8) [1d6bad4c] START, GetImagesListVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, sdUUID = 3573f707-d0c4-4dc9-b09f-dca9a3506a4f), log id: 44b9c1d1 >2014-09-11 14:18:27,986 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (ajp-/127.0.0.1:8702-8) [1d6bad4c] FINISH, GetImagesListVDSCommand, return: [], log id: 44b9c1d1 >2014-09-11 14:18:28,021 WARN [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (ajp-/127.0.0.1:8702-8) There are no OVF_STORE disks on storage domain id 3573f707-d0c4-4dc9-b09f-dca9a3506a4f >2014-09-11 14:18:28,031 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] Lock Acquired to object EngineLock [exclusiveLocks= key: 3573f707-d0c4-4dc9-b09f-dca9a3506a4f value: STORAGE >, sharedLocks= ] >2014-09-11 14:18:28,051 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] Running command: ActivateStorageDomainCommand internal: true. Entities affected : ID: 3573f707-d0c4-4dc9-b09f-dca9a3506a4f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:18:28,065 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] Lock freed to object EngineLock [exclusiveLocks= key: 3573f707-d0c4-4dc9-b09f-dca9a3506a4f value: STORAGE >, sharedLocks= ] >2014-09-11 14:18:28,065 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/11/14 2:18 PM >2014-09-11 14:18:28,071 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-12) [7c5ad46a] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-11 14:18:28,108 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-12) [7c5ad46a] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2d8ebe16 >2014-09-11 14:18:28,117 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-12) [7c5ad46a] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0}, log id: 2d8ebe16 >2014-09-11 14:18:28,123 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = 3573f707-d0c4-4dc9-b09f-dca9a3506a4f), log id: 4a4a358d >2014-09-11 14:18:28,403 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] FINISH, ActivateStorageDomainVDSCommand, return: true, log id: 4a4a358d >2014-09-11 14:18:28,403 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] ActivateStorage Domain. After Connect all hosts to pool. Time:9/11/14 2:18 PM >2014-09-11 14:18:28,445 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vdsId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 4d3274a1 >2014-09-11 14:18:28,661 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, ConnectStoragePoolVDSCommand, log id: 4d3274a1 >2014-09-11 14:18:28,662 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-6-thread-17) Refreshed vds rack in pool Default >2014-09-11 14:18:28,662 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [3ed1299] ActivateStorage Domain. After change storage pool status in vds. Time:9/11/14 2:18 PM >2014-09-11 14:18:28,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [3ed1299] Correlation ID: 3ed1299, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export (Data Center Default) was activated by <UNKNOWN> >2014-09-11 14:18:28,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [3ed1299] Correlation ID: 1d6bad4c, Job ID: 25de240c-e5b4-4fc8-9224-46d21d40fd6c, Call Stack: null, Custom Event ID: -1, Message: Storage Domain export was attached to Data Center Default by admin >2014-09-11 14:47:26,023 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-8) [2c81dd15] Lock Acquired to object EngineLock [exclusiveLocks= key: el6.5 value: VM_NAME >, sharedLocks= ] >2014-09-11 14:47:26,128 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-8) [2c81dd15] Running command: AddVmFromScratchCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_VM with role type USER >2014-09-11 14:47:26,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [2c81dd15] Correlation ID: 2c81dd15, Job ID: 71b6d50e-bd07-4350-bd77-91e931c6cd27, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 was created by admin. >2014-09-11 14:47:26,646 INFO [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (ajp-/127.0.0.1:8702-8) [2c81dd15] Lock freed to object EngineLock [exclusiveLocks= key: el6.5 value: VM_NAME >, sharedLocks= ] >2014-09-11 14:47:34,720 INFO [org.ovirt.engine.core.bll.network.vm.AddVmInterfaceCommand] (ajp-/127.0.0.1:8702-4) [1de228bd] Running command: AddVmInterfaceCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONFIGURE_VM_NETWORK with role type USER, ID: 0000000e-000e-000e-000e-0000000000e4 Type: VnicProfileAction group CONFIGURE_VM_NETWORK with role type USER >2014-09-11 14:47:34,870 INFO [org.ovirt.engine.core.bll.network.vm.ActivateDeactivateVmNicCommand] (ajp-/127.0.0.1:8702-4) [4d8ea393] Running command: ActivateDeactivateVmNicCommand internal: true. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VM >2014-09-11 14:47:35,046 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4d8ea393] Correlation ID: 4d8ea393, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 (VirtIO) was plugged to VM el6.5. (User: admin) >2014-09-11 14:47:35,076 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [4d8ea393] Correlation ID: 1de228bd, Call Stack: null, Custom Event ID: -1, Message: Interface nic1 (VirtIO) was added to VM el6.5. (User: admin) >2014-09-11 14:47:36,026 INFO [org.ovirt.engine.core.bll.network.vm.ReorderVmNicsCommand] (ajp-/127.0.0.1:8702-3) [4c0faee5] Running command: ReorderVmNicsCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CREATE_VM with role type USER >2014-09-11 14:47:36,111 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [4c0faee5] Correlation ID: 4c0faee5, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-11 14:47:44,005 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-10) [5209d01e] Lock Acquired to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-11 14:47:44,057 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (ajp-/127.0.0.1:8702-10) [5209d01e] Running command: AddDiskCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONFIGURE_VM_STORAGE with role type USER, ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group CREATE_DISK with role type USER >2014-09-11 14:47:44,168 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-10) [607d54a3] Running command: AddImageFromScratchCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 14:47:44,243 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (ajp-/127.0.0.1:8702-10) [607d54a3] Lock freed to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-11 14:47:44,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-10) [607d54a3] START, CreateImageVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 021e5bea-9f8e-446d-b19c-fb68839d4972, imageSizeInBytes = 21474836480, volumeFormat = RAW, newImageId = 7172d947-e2c8-4595-80bf-933c22f83b06, newImageDescription = ), log id: 48b6f3e4 >2014-09-11 14:47:44,247 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-10) [607d54a3] -- executeIrsBrokerCommand: calling 'createVolume' with two new parameters: description and UUID >2014-09-11 14:47:45,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateImageVDSCommand] (ajp-/127.0.0.1:8702-10) [607d54a3] FINISH, CreateImageVDSCommand, return: 7172d947-e2c8-4595-80bf-933c22f83b06, log id: 48b6f3e4 >2014-09-11 14:47:45,724 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-10) [607d54a3] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 74022fb8-62a6-4b18-84a4-8cd9e6c9f64e >2014-09-11 14:47:45,724 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-10) [607d54a3] CommandMultiAsyncTasks::AttachTask: Attaching task 79d1d7ca-80cd-46ed-88db-adef64cd142a to command 74022fb8-62a6-4b18-84a4-8cd9e6c9f64e. >2014-09-11 14:47:45,771 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-10) [607d54a3] Adding task 79d1d7ca-80cd-46ed-88db-adef64cd142a (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-11 14:47:45,877 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [607d54a3] Correlation ID: 5209d01e, Job ID: 5394508f-045a-475b-8729-71438d87e696, Call Stack: null, Custom Event ID: -1, Message: Add-Disk operation of el6.5_Disk1 was initiated on VM el6.5 by admin. >2014-09-11 14:47:45,877 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-10) [607d54a3] BaseAsyncTask::startPollingTask: Starting to poll task 79d1d7ca-80cd-46ed-88db-adef64cd142a. >2014-09-11 14:47:50,995 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-10) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-09-11 14:47:51,000 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-10) SPMAsyncTask::PollTask: Polling task 79d1d7ca-80cd-46ed-88db-adef64cd142a (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status running. >2014-09-11 14:47:51,001 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-10) Finished polling Tasks, will poll again in 10 seconds. >2014-09-11 14:48:01,007 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-35) SPMAsyncTask::PollTask: Polling task 79d1d7ca-80cd-46ed-88db-adef64cd142a (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-11 14:48:01,016 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-35) BaseAsyncTask::onTaskEndSuccess: Task 79d1d7ca-80cd-46ed-88db-adef64cd142a (Parent Command AddDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-11 14:48:01,017 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-35) CommandAsyncTask::endActionIfNecessary: All tasks of command 74022fb8-62a6-4b18-84a4-8cd9e6c9f64e has ended -> executing endAction >2014-09-11 14:48:01,017 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-35) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 74022fb8-62a6-4b18-84a4-8cd9e6c9f64e): calling endAction . >2014-09-11 14:48:01,018 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction AddDisk, executionIndex: 0 >2014-09-11 14:48:01,051 INFO [org.ovirt.engine.core.bll.AddDiskCommand] (org.ovirt.thread.pool-6-thread-23) [5209d01e] Ending command successfully: org.ovirt.engine.core.bll.AddDiskCommand >2014-09-11 14:48:01,080 INFO [org.ovirt.engine.core.bll.AddImageFromScratchCommand] (org.ovirt.thread.pool-6-thread-23) [607d54a3] Ending command successfully: org.ovirt.engine.core.bll.AddImageFromScratchCommand >2014-09-11 14:48:01,098 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-23) [607d54a3] START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 021e5bea-9f8e-446d-b19c-fb68839d4972, imageId = 7172d947-e2c8-4595-80bf-933c22f83b06), log id: 55eddaaa >2014-09-11 14:48:01,358 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-23) [607d54a3] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@9e6aa203, log id: 55eddaaa >2014-09-11 14:48:01,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-23) Correlation ID: 5209d01e, Call Stack: null, Custom Event ID: -1, Message: The disk el6.5_Disk1 was successfully added to VM el6.5. >2014-09-11 14:48:01,427 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk completed, handling the result. >2014-09-11 14:48:01,427 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type AddDisk succeeded, clearing tasks. >2014-09-11 14:48:01,439 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 79d1d7ca-80cd-46ed-88db-adef64cd142a >2014-09-11 14:48:01,439 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 79d1d7ca-80cd-46ed-88db-adef64cd142a), log id: 7ad0aa59 >2014-09-11 14:48:01,443 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=79d1d7ca-80cd-46ed-88db-adef64cd142a), log id: 6b962a7e >2014-09-11 14:48:01,835 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) FINISH, HSMClearTaskVDSCommand, log id: 6b962a7e >2014-09-11 14:48:01,835 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) FINISH, SPMClearTaskVDSCommand, log id: 7ad0aa59 >2014-09-11 14:48:01,858 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) BaseAsyncTask::removeTaskFromDB: Removed task 79d1d7ca-80cd-46ed-88db-adef64cd142a from DataBase >2014-09-11 14:48:01,858 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 74022fb8-62a6-4b18-84a4-8cd9e6c9f64e >2014-09-11 14:48:06,066 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [236fb84] Lock Acquired to object EngineLock [exclusiveLocks= key: el6.5 value: VM_NAME >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-11 14:48:06,196 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [236fb84] Running command: UpdateVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group EDIT_VM_PROPERTIES with role type USER >2014-09-11 14:48:06,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [236fb84] Correlation ID: 236fb84, Job ID: 36b9f09c-7ea7-47bf-a12f-96267ae7c2e9, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 configuration was updated by admin. >2014-09-11 14:48:06,325 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (ajp-/127.0.0.1:8702-9) [236fb84] Lock freed to object EngineLock [exclusiveLocks= key: el6.5 value: VM_NAME >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-11 14:48:14,922 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-7) [478d7346] Lock Acquired to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-11 14:48:14,971 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-7) [478d7346] START, IsVmDuringInitiatingVDSCommand( vmId = 14034173-04d2-4b26-b4da-42ce14e33e13), log id: 3b962104 >2014-09-11 14:48:14,971 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-7) [478d7346] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 3b962104 >2014-09-11 14:48:15,148 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] Running command: RunVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-11 14:48:15,242 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] START, UpdateVmDynamicDataVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@e257eb8), log id: 646d3e47 >2014-09-11 14:48:15,253 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] FINISH, UpdateVmDynamicDataVDSCommand, log id: 646d3e47 >2014-09-11 14:48:15,273 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] START, CreateVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: 78e1826d >2014-09-11 14:48:15,385 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] START, CreateVmFromCloudInitVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: 62f701d4 >2014-09-11 14:48:15,616 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=14034173-04d2-4b26-b4da-42ce14e33e13,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=5120,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{shared=false, iface=ide, index=3, specParams={vmPayload={file={openstack/latest/meta_data.json=ewogICJsYXVuY2hfaW5kZXgiIDogIjAiLAogICJhdmFpbGFiaWxpdHlfem9uZSIgOiAibm92YSIs >CiAgInV1aWQiIDogIjIyODFiYTNhLWMwMjgtNDNiZi1hYjdmLTAzYWFhMTdhNmZlNiIsCiAgIm1l >dGEiIDogewogICAgImVzc2VudGlhbCIgOiAiZmFsc2UiLAogICAgInJvbGUiIDogInNlcnZlciIs >CiAgICAiZHNtb2RlIiA6ICJsb2NhbCIKICB9Cn0= >, openstack/latest/user_data=I2Nsb3VkLWNvbmZpZwpzc2hfcHdhdXRoOiB0cnVlCmRpc2FibGVfcm9vdDogMApvdXRwdXQ6CiAg >YWxsOiAnPj4gL3Zhci9sb2cvY2xvdWQtaW5pdC1vdXRwdXQubG9nJwpjaHBhc3N3ZDoKICBleHBp >cmU6IGZhbHNlCnJ1bmNtZDoKLSAnc2VkIC1pICcnL15kYXRhc291cmNlX2xpc3Q6IC9kJycgL2V0 >Yy9jbG91ZC9jbG91ZC5jZmc7IGVjaG8gJydkYXRhc291cmNlX2xpc3Q6CiAgWyJOb0Nsb3VkIiwg >IkNvbmZpZ0RyaXZlIl0nJyA+PiAvZXRjL2Nsb3VkL2Nsb3VkLmNmZycK >}, volId=config-2}}, path=, device=cdrom, type=disk, readonly=true, deviceId=ec17508d-1590-425b-bcde-d03e447107ac}, {specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=06d56511-11b1-42cb-bf27-371743a81867}, {shared=false, iface=ide, index=2, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=3f23874c-1813-4b86-a266-fd6e5916fa77}, {shared=false, index=0, volumeID=7172d947-e2c8-4595-80bf-933c22f83b06, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=021e5bea-9f8e-446d-b19c-fb68839d4972, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=021e5bea-9f8e-446d-b19c-fb68839d4972}, {nicModel=pv, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:91, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92461e66-f605-47da-8c88-9f857f128b3c}, {specParams={model=virtio}, device=memballoon, type=balloon, deviceId=ce3f7b1c-f540-4d2d-b285-769eb9e05542}, {index=0, model=virtio-scsi, specParams={}, device=scsi, type=controller, deviceId=fbd9cae2-f563-4db3-b236-4e763aa059fe}, {specParams={}, device=virtio-serial, type=controller, deviceId=f005d666-1889-4213-b96b-e5f48281f18e}],vmName=el6.5,cpuType=SandyBridge,fileTransferEnable=true >2014-09-11 14:48:15,638 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVmFromCloudInitVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] FINISH, CreateVmFromCloudInitVDSCommand, log id: 62f701d4 >2014-09-11 14:48:15,687 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 78e1826d >2014-09-11 14:48:15,687 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-35) [478d7346] Lock freed to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-11 14:48:15,721 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [478d7346] Correlation ID: 478d7346, Job ID: c6e4d36b-d998-4a62-8625-caf66b5acafe, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 was started by admin (Host: rack). >2014-09-11 14:48:18,268 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from WaitForLaunch --> PoweringUp >2014-09-11 14:48:18,270 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 144e8522 >2014-09-11 14:48:18,299 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-75) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=23703, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5903, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=Up, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5902, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@8f11be2, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}, {acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@d5d9704, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 144e8522 >2014-09-11 14:48:18,302 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {tlsPort=5903, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5902} >2014-09-11 14:48:18,322 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-11 14:48:18,322 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-11 14:48:21,009 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-9) [7d71678e] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 14:48:21,030 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [7d71678e] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=Q5qg8H8dZBxb, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: ff8c743 >2014-09-11 14:48:21,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [7d71678e] FINISH, SetVmTicketVDSCommand, log id: ff8c743 >2014-09-11 14:48:21,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7d71678e] Correlation ID: 7d71678e, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 14:48:28,796 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-3) [6502136b] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 14:48:28,811 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-3) [6502136b] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=5KUhk4olRsAS, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: a23bcb7 >2014-09-11 14:48:28,851 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-3) [6502136b] FINISH, SetVmTicketVDSCommand, log id: a23bcb7 >2014-09-11 14:48:28,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [6502136b] Correlation ID: 6502136b, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 14:48:38,514 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-11) [160eb684] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 14:48:38,527 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [160eb684] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=U0AYpPPqEHX5, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 2603f422 >2014-09-11 14:48:38,569 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [160eb684] FINISH, SetVmTicketVDSCommand, log id: 2603f422 >2014-09-11 14:48:38,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [160eb684] Correlation ID: 160eb684, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 14:48:49,152 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-11 14:48:49,154 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] Entered (VM f20). >2014-09-11 14:48:49,154 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] Sending shutdown command for VM f20. >2014-09-11 14:48:49,157 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 5847689d >2014-09-11 14:48:49,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 7d346cf0 >2014-09-11 14:48:49,169 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] FINISH, DestroyVDSCommand, log id: 7d346cf0 >2014-09-11 14:48:49,184 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 5847689d >2014-09-11 14:48:49,198 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-13) [7d791a2a] Correlation ID: 7d791a2a, Job ID: 62e48ba4-60a9-4977-a172-5153c0c17ce4, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM f20 (Host: rack) (Reason: Not Specified). >2014-09-11 14:49:01,616 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-6-thread-11) [1027401b] Running command: StopVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-11 14:49:01,622 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-11) [1027401b] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=0, gracefully=false, reason=), log id: 5cc7822 >2014-09-11 14:49:01,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-11) [1027401b] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=0, gracefully=false, reason=), log id: 38d2aa04 >2014-09-11 14:49:02,088 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-11) [1027401b] FINISH, DestroyVDSCommand, log id: 38d2aa04 >2014-09-11 14:49:02,110 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-11) [1027401b] FINISH, DestroyVmVDSCommand, return: Down, log id: 5cc7822 >2014-09-11 14:49:02,112 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-29) [108b266f] Running command: ProcessDownVmCommand internal: true. >2014-09-11 14:49:02,149 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-11) [1027401b] Correlation ID: 1027401b, Job ID: 5d4f43e3-4ebf-4b7d-b02b-b3d491b74daf, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 powered off by admin (Host: rack) (Reason: Not Specified). >2014-09-11 14:49:06,665 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-6) [2d38c53b] Lock Acquired to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-11 14:49:06,733 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [2d38c53b] START, IsVmDuringInitiatingVDSCommand( vmId = 14034173-04d2-4b26-b4da-42ce14e33e13), log id: 252611d >2014-09-11 14:49:06,734 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [2d38c53b] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 252611d >2014-09-11 14:49:06,815 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] Running command: RunVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-11 14:49:06,935 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] START, UpdateVmDynamicDataVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@8406a70b), log id: bc1e095 >2014-09-11 14:49:06,945 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] FINISH, UpdateVmDynamicDataVDSCommand, log id: bc1e095 >2014-09-11 14:49:06,963 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] START, CreateVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: 41586e2b >2014-09-11 14:49:06,980 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] START, CreateVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: 19ac54ce >2014-09-11 14:49:07,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=14034173-04d2-4b26-b4da-42ce14e33e13,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=5120,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, type=pci, slot=0x02, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=06d56511-11b1-42cb-bf27-371743a81867}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=3f23874c-1813-4b86-a266-fd6e5916fa77}, {shared=false, index=0, volumeID=7172d947-e2c8-4595-80bf-933c22f83b06, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=021e5bea-9f8e-446d-b19c-fb68839d4972, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=021e5bea-9f8e-446d-b19c-fb68839d4972}, {nicModel=pv, address={bus=0x00, domain=0x0000, type=pci, slot=0x03, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:91, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92461e66-f605-47da-8c88-9f857f128b3c}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x07, function=0x0}, specParams={model=virtio}, device=memballoon, type=balloon, deviceId=ce3f7b1c-f540-4d2d-b285-769eb9e05542}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=fbd9cae2-f563-4db3-b236-4e763aa059fe}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=f005d666-1889-4213-b96b-e5f48281f18e}],vmName=el6.5,cpuType=SandyBridge,fileTransferEnable=true >2014-09-11 14:49:07,107 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] FINISH, CreateVDSCommand, log id: 19ac54ce >2014-09-11 14:49:07,119 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 41586e2b >2014-09-11 14:49:07,119 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] Lock freed to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-11 14:49:07,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-39) [2d38c53b] Correlation ID: 2d38c53b, Job ID: 438c58a0-d93f-40e4-a5b3-86155dfe731c, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 was started by admin (Host: rack). >2014-09-11 14:49:07,266 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 413cbc94 >2014-09-11 14:49:07,280 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=0, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=WaitForLaunch, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@1933d67c, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}, {acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@1a4673d1, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 413cbc94 >2014-09-11 14:49:07,283 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, type=graphics} >2014-09-11 14:49:07,285 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-11 14:49:07,285 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-11 14:49:09,756 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-83) Setting new tasks map. The map contains now 0 tasks >2014-09-11 14:49:09,756 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-83) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-11 14:49:10,559 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-76) Unexpected return value: StatusForXmlRpc [mCode=-32603, mMessage='vnet1'] >2014-09-11 14:49:10,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-76) Failed in GetStatsVDS method >2014-09-11 14:49:10,567 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand] (DefaultQuartzScheduler_Worker-76) Command GetStatsVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288]) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetStatsVDS, error = 'vnet1', code = -32603 >2014-09-11 14:49:10,570 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) vds::refreshVdsStats Failed getVdsStats, vds = f689c28a-07de-4408-ac86-81382b6dd288 : rack, error = org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetStatsVDS, error = 'vnet1', code = -32603 >2014-09-11 14:49:10,570 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-76) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetStatsVDS, error = 'vnet1', code = -32603 > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:61) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:200) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:187) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:680) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:634) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-11 14:49:10,576 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-76) Failed to refresh VDS , vds = f689c28a-07de-4408-ac86-81382b6dd288 : rack, error = org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetStatsVDS, error = 'vnet1', code = -32603, continuing.: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetStatsVDS, error = 'vnet1', code = -32603 > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:61) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:200) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:187) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsVDSCommand.executeVdsBrokerCommand(GetStatsVDSCommand.java:24) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsStats(VdsUpdateRunTimeInfo.java:680) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:634) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-11 14:49:13,598 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) [18c02297] VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from WaitForLaunch --> PoweringUp >2014-09-11 14:49:13,598 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) [18c02297] START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 63c635bc >2014-09-11 14:49:13,614 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-7) [18c02297] FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=24809, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5903, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=Up, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5902, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@69f1fcd0, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}, {acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@e0cf207, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 63c635bc >2014-09-11 14:49:13,617 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) [18c02297] Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {tlsPort=5903, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5902} >2014-09-11 14:49:13,620 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) [18c02297] Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-11 14:49:13,620 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-7) [18c02297] Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-11 14:49:16,577 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-10) [1db1f10e] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 14:49:16,590 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-10) [1db1f10e] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=+oMxdVvc2Wcf, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 49b17e61 >2014-09-11 14:49:16,624 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-10) [1db1f10e] FINISH, SetVmTicketVDSCommand, log id: 49b17e61 >2014-09-11 14:49:16,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) [1db1f10e] Correlation ID: 1db1f10e, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 14:49:26,145 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-9) [76d608a3] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 14:49:26,158 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [76d608a3] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=weGkIlsXFANt, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 1c529273 >2014-09-11 14:49:26,192 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [76d608a3] FINISH, SetVmTicketVDSCommand, log id: 1c529273 >2014-09-11 14:49:26,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [76d608a3] Correlation ID: 76d608a3, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 14:49:32,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-41) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM el6.5. >2014-09-11 14:49:44,312 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-73) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM el6.5. >2014-09-11 14:50:08,752 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from PoweringUp --> Up >2014-09-11 14:50:08,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: 2d38c53b, Job ID: 438c58a0-d93f-40e4-a5b3-86155dfe731c, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 started on Host rack >2014-09-11 14:53:52,035 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from PoweringDown --> Up >2014-09-11 14:53:52,057 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-30) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Shutdown of VM f20 failed. >2014-09-11 14:54:39,676 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-09-11 14:54:47,382 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) Attempting to update VMs/Templates Ovf. >2014-09-11 14:54:47,823 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 14:54:47,828 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 14:54:47,848 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: af849d >2014-09-11 14:54:47,849 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 14:54:47,849 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 10:54:40 CEST 2014"} > >2014-09-11 14:54:48,317 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [6dc1f3fb] FINISH, SetVolumeDescriptionVDSCommand, log id: af849d >2014-09-11 14:54:48,418 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 14:54:48,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 4a84bcd9 >2014-09-11 14:54:48,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- executeVdsBrokerCommand: >2014-09-11 14:54:48,421 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=20,480 > >2014-09-11 14:54:51,560 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] FINISH, UploadStreamVDSCommand, log id: 4a84bcd9 >2014-09-11 14:54:51,564 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-52) [26e10553] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 11c5a432-467e-4e9f-8da3-ae19db6ea711 >2014-09-11 14:54:51,564 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-52) [26e10553] CommandMultiAsyncTasks::AttachTask: Attaching task 5ba6b20c-9229-4cb4-af6b-21031efee3cf to command 11c5a432-467e-4e9f-8da3-ae19db6ea711. >2014-09-11 14:54:51,583 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-52) [26e10553] Adding task 5ba6b20c-9229-4cb4-af6b-21031efee3cf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-11 14:54:51,599 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 14:54:51,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 1ddffd53 >2014-09-11 14:54:51,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 14:54:51,617 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 14:54:47 CEST 2014","Size":20480} > >2014-09-11 14:54:52,211 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] FINISH, SetVolumeDescriptionVDSCommand, log id: 1ddffd53 >2014-09-11 14:54:52,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 33d1c9da >2014-09-11 14:54:52,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 14:54:52,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 10:54:40 CEST 2014"} > >2014-09-11 14:54:52,699 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [26e10553] FINISH, SetVolumeDescriptionVDSCommand, log id: 33d1c9da >2014-09-11 14:54:52,748 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-11 14:54:52,750 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] START, UploadStreamVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 16ec34bc >2014-09-11 14:54:52,750 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] -- executeVdsBrokerCommand: >2014-09-11 14:54:52,750 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=20,480 > >2014-09-11 14:54:55,082 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] FINISH, UploadStreamVDSCommand, log id: 16ec34bc >2014-09-11 14:54:55,086 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-52) [396e9d49] CommandMultiAsyncTasks::AttachTask: Attaching task b267672b-954e-45ee-a454-5f7bd90493b0 to command 11c5a432-467e-4e9f-8da3-ae19db6ea711. >2014-09-11 14:54:55,111 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Adding task b267672b-954e-45ee-a454-5f7bd90493b0 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-11 14:54:55,124 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 14:54:55,124 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 14:54:55,124 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 14:54:55,125 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 14:54:55,125 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 14:54:55,128 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: ec5a9b7 >2014-09-11 14:54:55,128 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-11 14:54:55,129 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 14:54:47 CEST 2014","Size":20480} > >2014-09-11 14:54:55,483 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-52) [396e9d49] FINISH, SetVolumeDescriptionVDSCommand, log id: ec5a9b7 >2014-09-11 14:54:55,488 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-52) [396e9d49] BaseAsyncTask::startPollingTask: Starting to poll task 5ba6b20c-9229-4cb4-af6b-21031efee3cf. >2014-09-11 14:54:55,488 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-52) [396e9d49] BaseAsyncTask::startPollingTask: Starting to poll task b267672b-954e-45ee-a454-5f7bd90493b0. >2014-09-11 14:55:01,059 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-09-11 14:55:01,065 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task 5ba6b20c-9229-4cb4-af6b-21031efee3cf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-11 14:55:01,066 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task 5ba6b20c-9229-4cb4-af6b-21031efee3cf (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-11 14:55:01,066 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-88) Task with DB Task ID 9c4275f2-96e1-4792-ad4e-efad2e8028a8 and VDSM Task ID b267672b-954e-45ee-a454-5f7bd90493b0 is in state Polling. End action for command 11c5a432-467e-4e9f-8da3-ae19db6ea711 will proceed when all the entitys tasks are completed. >2014-09-11 14:55:01,066 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) SPMAsyncTask::PollTask: Polling task b267672b-954e-45ee-a454-5f7bd90493b0 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-11 14:55:01,067 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-88) BaseAsyncTask::onTaskEndSuccess: Task b267672b-954e-45ee-a454-5f7bd90493b0 (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-11 14:55:01,068 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endActionIfNecessary: All tasks of command 11c5a432-467e-4e9f-8da3-ae19db6ea711 has ended -> executing endAction >2014-09-11 14:55:01,068 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-88) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 11c5a432-467e-4e9f-8da3-ae19db6ea711): calling endAction . >2014-09-11 14:55:01,068 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-09-11 14:55:01,104 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-09-11 14:55:01,105 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-11 14:55:01,105 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-11 14:55:01,105 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-11 14:55:01,105 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-11 14:55:01,106 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-11 14:55:01,106 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-09-11 14:55:01,106 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-09-11 14:55:01,106 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 5ba6b20c-9229-4cb4-af6b-21031efee3cf >2014-09-11 14:55:01,106 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 5ba6b20c-9229-4cb4-af6b-21031efee3cf), log id: 5ad55fca >2014-09-11 14:55:01,109 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=5ba6b20c-9229-4cb4-af6b-21031efee3cf), log id: 6e83f55c >2014-09-11 14:55:01,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] FINISH, HSMClearTaskVDSCommand, log id: 6e83f55c >2014-09-11 14:55:01,253 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] FINISH, SPMClearTaskVDSCommand, log id: 5ad55fca >2014-09-11 14:55:01,263 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] BaseAsyncTask::removeTaskFromDB: Removed task 5ba6b20c-9229-4cb4-af6b-21031efee3cf from DataBase >2014-09-11 14:55:01,263 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] SPMAsyncTask::ClearAsyncTask: Attempting to clear task b267672b-954e-45ee-a454-5f7bd90493b0 >2014-09-11 14:55:01,263 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = b267672b-954e-45ee-a454-5f7bd90493b0), log id: 4dac4964 >2014-09-11 14:55:01,265 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] START, HSMClearTaskVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, taskId=b267672b-954e-45ee-a454-5f7bd90493b0), log id: 6a066c12 >2014-09-11 14:55:01,470 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] FINISH, HSMClearTaskVDSCommand, log id: 6a066c12 >2014-09-11 14:55:01,470 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] FINISH, SPMClearTaskVDSCommand, log id: 4dac4964 >2014-09-11 14:55:01,479 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] BaseAsyncTask::removeTaskFromDB: Removed task b267672b-954e-45ee-a454-5f7bd90493b0 from DataBase >2014-09-11 14:55:01,480 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-23) [6dc1f3fb] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 11c5a432-467e-4e9f-8da3-ae19db6ea711 >2014-09-11 14:56:09,770 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-48) Setting new tasks map. The map contains now 0 tasks >2014-09-11 14:56:09,770 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-48) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-11 14:57:57,362 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from Up --> RebootInProgress >2014-09-11 14:57:57,362 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 6b2d2c7c >2014-09-11 14:57:57,378 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-93) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=24809, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5903, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=Up, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5902, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@62c3b7dc, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}, {acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@1f5bff2c, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 6b2d2c7c >2014-09-11 14:57:57,384 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {tlsPort=5903, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5902} >2014-09-11 14:57:57,390 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-11 14:57:57,390 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-93) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-11 14:58:58,600 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from RebootInProgress --> Up >2014-09-11 15:54:39,695 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-63) Principal admin::internal synchronized >2014-09-11 15:54:55,489 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-8) Attempting to update VMs/Templates Ovf. >2014-09-11 16:54:39,710 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-76) Principal admin::internal synchronized >2014-09-11 16:54:55,524 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) [3de201f6] Attempting to update VMs/Templates Ovf. >2014-09-11 16:55:40,425 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-8) [56781881] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-11 16:55:40,440 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-8) [56781881] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=8Fwija4g0fXR, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 43ba73a0 >2014-09-11 16:55:40,480 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-8) [56781881] FINISH, SetVmTicketVDSCommand, log id: 43ba73a0 >2014-09-11 16:55:40,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [56781881] Correlation ID: 56781881, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-11 16:55:43,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-42) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM el6.5. >2014-09-11 17:00:27,660 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-09-11 17:00:27,668 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 17:05:10,585 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) START, GetVmsInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = 3573f707-d0c4-4dc9-b09f-dca9a3506a4f, vmIdList = null), log id: 788299f0 >2014-09-11 17:05:10,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-2) FINISH, GetVmsInfoVDSCommand, log id: 788299f0 >2014-09-11 17:05:12,347 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) START, GetVmsInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = 3573f707-d0c4-4dc9-b09f-dca9a3506a4f, vmIdList = null), log id: 5c67203e >2014-09-11 17:05:12,403 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVmsInfoVDSCommand] (ajp-/127.0.0.1:8702-1) FINISH, GetVmsInfoVDSCommand, log id: 5c67203e >2014-09-11 17:09:23,531 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [4b5a35c4] Running command: LogoutUserCommand internal: false. >2014-09-11 17:09:23,550 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [4b5a35c4] Correlation ID: 4b5a35c4, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 17:09:23,626 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-11 17:09:23,626 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-11 17:10:47,242 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginAdminUserCommand internal: false. >2014-09-11 17:10:47,252 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 17:12:43,182 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [5bf76aa9] Running command: LogoutUserCommand internal: false. >2014-09-11 17:12:43,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [5bf76aa9] Correlation ID: 5bf76aa9, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 17:12:43,378 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-11 17:12:43,379 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-11 17:13:04,875 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-11 17:13:04,876 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-11 17:13:13,353 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginAdminUserCommand internal: false. >2014-09-11 17:13:13,365 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 17:13:13,463 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2014-09-11 17:13:25,179 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-09-11 17:13:25,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 17:13:25,314 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-6) [30cc9b72] Running command: LogoutBySessionCommand internal: false. >2014-09-11 17:13:25,340 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [364aec89] Running command: LogoutUserCommand internal: false. >2014-09-11 17:13:25,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [364aec89] Correlation ID: 364aec89, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-11 17:13:25,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) Correlation ID: 30cc9b72, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-11 17:13:30,187 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-11 17:13:30,187 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-11 17:14:35,559 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-9) Running command: LoginUserCommand internal: false. >2014-09-11 17:14:35,595 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-11 17:25:34,937 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-11 17:25:34,937 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-11 17:54:39,722 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-11 17:54:55,553 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-11 17:56:17,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-43) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM el6.5. >2014-09-11 18:54:39,740 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-11 18:54:55,590 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-09-11 19:54:39,753 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-11 19:54:55,621 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-15) Attempting to update VMs/Templates Ovf. >2014-09-11 20:54:39,764 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Principal admin::internal synchronized >2014-09-11 20:54:55,647 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2014-09-11 21:54:39,778 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-49) Principal admin::internal synchronized >2014-09-11 21:54:55,683 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-11 22:54:39,791 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-11 22:54:55,705 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-15) Attempting to update VMs/Templates Ovf. >2014-09-11 23:54:39,803 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-19) Principal admin::internal synchronized >2014-09-11 23:54:55,728 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-12 00:54:39,814 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-12 00:54:55,761 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) [18c02297] Attempting to update VMs/Templates Ovf. >2014-09-12 01:54:39,835 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-09-12 01:54:55,795 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-12 02:54:39,849 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-30) Principal admin::internal synchronized >2014-09-12 02:54:55,816 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-09-12 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-42) Start deleteAgedOutAuditLogs >2014-09-12 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-85) Start deleteAgedOutCommandEntities >2014-09-12 03:35:35,013 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-42) Finished deleteAgedOutAuditLogs >2014-09-12 03:35:35,017 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-85) Finished deleteAgedOutCommandEntities >2014-09-12 03:54:39,861 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-49) Principal admin::internal synchronized >2014-09-12 03:54:55,843 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-12 04:54:39,871 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) Principal admin::internal synchronized >2014-09-12 04:54:55,875 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-15) Attempting to update VMs/Templates Ovf. >2014-09-12 05:54:39,882 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-86) Principal admin::internal synchronized >2014-09-12 05:54:55,898 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2014-09-12 06:54:39,894 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-12 06:54:55,919 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-12 07:54:39,906 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-12 07:54:55,938 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-09-12 08:54:39,917 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-12 08:54:55,961 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-12 09:30:32,192 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-7) Running command: LoginUserCommand internal: false. >2014-09-12 09:30:32,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-12 09:31:04,743 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginAdminUserCommand internal: false. >2014-09-12 09:31:04,758 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-12 09:54:39,928 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-62) Principal admin::internal synchronized >2014-09-12 09:54:55,987 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-12 10:49:05,228 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-12 10:49:05,231 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] Entered (VM f20). >2014-09-12 10:49:05,232 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] Sending shutdown command for VM f20. >2014-09-12 10:49:05,235 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 3299369 >2014-09-12 10:49:05,244 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 557257b0 >2014-09-12 10:49:05,249 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] FINISH, DestroyVDSCommand, log id: 557257b0 >2014-09-12 10:49:05,268 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-43) [2acae00a] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 3299369 >2014-09-12 10:49:05,286 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-43) [2acae00a] Correlation ID: 2acae00a, Job ID: c2ece448-e223-443b-b939-c4f487a60c02, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM f20 (Host: rack) (Reason: Not Specified). >2014-09-12 10:49:22,673 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-12 10:49:22,678 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] Entered (VM el6.5). >2014-09-12 10:49:22,679 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] Sending shutdown command for VM el6.5. >2014-09-12 10:49:22,682 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=30, gracefully=true, reason=), log id: 676552 >2014-09-12 10:49:22,738 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=30, gracefully=true, reason=), log id: 43976659 >2014-09-12 10:49:22,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] FINISH, DestroyVDSCommand, log id: 43976659 >2014-09-12 10:49:22,769 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 676552 >2014-09-12 10:49:22,797 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-23) [4793bb0d] Correlation ID: 4793bb0d, Job ID: 434b2352-834a-4e2f-80d2-4654c5e27000, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM el6.5 (Host: rack) (Reason: Not Specified). >2014-09-12 10:49:36,044 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-2) START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=0, gracefully=false, reason=), log id: 1bcd56c2 >2014-09-12 10:49:36,047 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-2) FINISH, DestroyVDSCommand, log id: 1bcd56c2 >2014-09-12 10:49:36,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 is down. Exit message: Admin shut down from the engine >2014-09-12 10:49:36,074 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) VM el6.5 (14034173-04d2-4b26-b4da-42ce14e33e13) is running in db and not running in VDS rack >2014-09-12 10:49:36,074 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-2) START, FullListVdsCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vds=Host[rack,f689c28a-07de-4408-ac86-81382b6dd288], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 4dafa9dd >2014-09-12 10:49:36,086 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-2) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=4262, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=SandyBridge, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.62.208, displayPort=5900, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@47b91403, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 4dafa9dd >2014-09-12 10:49:36,090 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-12 10:49:36,090 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-2) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {tlsPort=5901, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.62.208, displayNetwork=rhevm}, deviceType=graphics, device=spice, type=graphics, port=5900} >2014-09-12 10:49:36,200 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-40) [3ef3f0df] Running command: ProcessDownVmCommand internal: true. >2014-09-12 10:54:07,618 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-53) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from PoweringDown --> Up >2014-09-12 10:54:07,694 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-53) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Shutdown of VM f20 failed. >2014-09-12 10:54:39,941 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) Principal admin::internal synchronized >2014-09-12 10:54:56,013 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-12 10:55:18,196 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-2) [30826ad5] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-12 10:55:18,213 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-2) [30826ad5] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=KMw7d8Y+MeNW, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 8a5de0a >2014-09-12 10:55:18,245 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-2) [30826ad5] FINISH, SetVmTicketVDSCommand, log id: 8a5de0a >2014-09-12 10:55:18,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [30826ad5] Correlation ID: 30826ad5, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-12 10:55:37,888 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-7) [6076552] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-12 10:55:37,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [6076552] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=vJL/ILFrFCO3, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 6ff761c8 >2014-09-12 10:55:37,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-7) [6076552] FINISH, SetVmTicketVDSCommand, log id: 6ff761c8 >2014-09-12 10:55:37,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [6076552] Correlation ID: 6076552, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-12 10:56:08,073 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-5) [7696ad32] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-12 10:56:08,088 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-5) [7696ad32] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=YiWg0rGA9Q40, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 4991e991 >2014-09-12 10:56:08,130 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-5) [7696ad32] FINISH, SetVmTicketVDSCommand, log id: 4991e991 >2014-09-12 10:56:08,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [7696ad32] Correlation ID: 7696ad32, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-12 10:56:17,631 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-11) [15560765] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-12 10:56:17,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [15560765] START, SetVmTicketVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=mbtVaiEkWqEV, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 4bc95243 >2014-09-12 10:56:17,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-11) [15560765] FINISH, SetVmTicketVDSCommand, log id: 4bc95243 >2014-09-12 10:56:17,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) [15560765] Correlation ID: 15560765, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-12 10:56:21,979 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM f20. >2014-09-12 10:56:31,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-45) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM f20. >2014-09-12 10:56:37,568 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-6-thread-25) [43bcf934] Running command: StopVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-12 10:56:37,573 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-25) [43bcf934] START, DestroyVmVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=0, gracefully=false, reason=), log id: 7e34db >2014-09-12 10:56:37,579 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-25) [43bcf934] START, DestroyVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=0, gracefully=false, reason=), log id: 488b48ba >2014-09-12 10:56:38,051 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-25) [43bcf934] FINISH, DestroyVDSCommand, log id: 488b48ba >2014-09-12 10:56:38,065 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-25) [43bcf934] FINISH, DestroyVmVDSCommand, return: Down, log id: 7e34db >2014-09-12 10:56:38,073 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-49) [20d0bda2] Running command: ProcessDownVmCommand internal: true. >2014-09-12 10:56:38,084 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-25) [43bcf934] Correlation ID: 43bcf934, Job ID: 5d474058-a2bb-4b09-be0a-cdd206294f7e, Call Stack: null, Custom Event ID: -1, Message: VM f20 powered off by admin (Host: rack) (Reason: Not Specified). >2014-09-12 11:54:39,954 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-59) Principal admin::internal synchronized >2014-09-12 11:54:56,027 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-09-12 12:54:39,964 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-78) Principal admin::internal synchronized >2014-09-12 12:54:56,054 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-09-12 13:54:39,975 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-9) Principal admin::internal synchronized >2014-09-12 13:54:56,078 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-75) Attempting to update VMs/Templates Ovf. >2014-09-12 14:54:39,990 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-57) Principal admin::internal synchronized >2014-09-12 14:54:56,102 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-12 15:54:40,001 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-12 15:54:56,121 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-12 16:54:40,012 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-20) [3de201f6] Principal admin::internal synchronized >2014-09-12 16:54:56,146 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-09-12 17:54:40,023 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-14) Principal admin::internal synchronized >2014-09-12 17:54:56,167 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) Attempting to update VMs/Templates Ovf. >2014-09-12 18:54:40,040 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-09-12 18:54:56,186 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [396e9d49] Attempting to update VMs/Templates Ovf. >2014-09-12 19:54:40,051 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-74) Principal admin::internal synchronized >2014-09-12 19:54:56,206 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) Attempting to update VMs/Templates Ovf. >2014-09-12 20:54:40,062 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-29) Principal admin::internal synchronized >2014-09-12 20:54:56,230 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2014-09-12 21:54:40,073 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-20) [3de201f6] Principal admin::internal synchronized >2014-09-12 21:54:56,252 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update VMs/Templates Ovf. >2014-09-12 22:54:40,086 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-24) Principal admin::internal synchronized >2014-09-12 22:54:56,279 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-09-12 23:54:40,099 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-12 23:54:56,314 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-13 00:54:40,113 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Principal admin::internal synchronized >2014-09-13 00:54:56,336 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-13 01:54:40,124 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-59) Principal admin::internal synchronized >2014-09-13 01:54:56,360 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-09-13 02:54:40,139 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-13 02:54:56,383 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) [18c02297] Attempting to update VMs/Templates Ovf. >2014-09-13 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-21) Start deleteAgedOutAuditLogs >2014-09-13 03:35:35,003 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-77) Start deleteAgedOutCommandEntities >2014-09-13 03:35:35,006 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-21) Finished deleteAgedOutAuditLogs >2014-09-13 03:35:35,011 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-77) Finished deleteAgedOutCommandEntities >2014-09-13 03:54:40,150 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) [18c02297] Principal admin::internal synchronized >2014-09-13 03:54:56,406 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-09-13 04:54:40,160 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-32) Principal admin::internal synchronized >2014-09-13 04:54:56,426 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-09-13 05:54:40,170 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-09-13 05:54:56,450 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-13 06:54:40,181 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-9) Principal admin::internal synchronized >2014-09-13 06:54:56,471 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-13 07:54:40,191 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-10) Principal admin::internal synchronized >2014-09-13 07:54:56,495 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-13 08:54:40,204 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-66) Principal admin::internal synchronized >2014-09-13 08:54:56,516 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-84) Attempting to update VMs/Templates Ovf. >2014-09-13 09:54:40,212 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-61) Principal admin::internal synchronized >2014-09-13 09:54:56,546 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) Attempting to update VMs/Templates Ovf. >2014-09-13 10:54:40,222 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-13 10:54:56,569 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) [3de201f6] Attempting to update VMs/Templates Ovf. >2014-09-13 11:54:40,232 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-51) Principal admin::internal synchronized >2014-09-13 11:54:56,593 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-13 12:54:40,242 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-15) Principal admin::internal synchronized >2014-09-13 12:54:56,613 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-13 13:54:40,251 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-13 13:54:56,639 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-09-13 14:54:40,267 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-13 14:54:56,663 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2014-09-13 15:54:40,276 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) Principal admin::internal synchronized >2014-09-13 15:54:56,685 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-09-13 16:54:40,287 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-13 16:54:56,706 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-09-13 17:54:40,296 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-13 17:54:56,728 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-13 18:54:40,305 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-13 18:54:56,747 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-13 19:54:40,315 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-85) Principal admin::internal synchronized >2014-09-13 19:54:56,770 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-09-13 20:54:40,324 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-2) Principal admin::internal synchronized >2014-09-13 20:54:56,794 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-65) Attempting to update VMs/Templates Ovf. >2014-09-13 21:54:40,334 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-71) Principal admin::internal synchronized >2014-09-13 21:54:56,814 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-09-13 22:54:40,344 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) Principal admin::internal synchronized >2014-09-13 22:54:56,834 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-09-13 23:54:40,354 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-13 23:54:56,857 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) Attempting to update VMs/Templates Ovf. >2014-09-14 00:54:40,370 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-89) Principal admin::internal synchronized >2014-09-14 00:54:56,879 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-09-14 01:54:40,379 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-32) Principal admin::internal synchronized >2014-09-14 01:54:56,901 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-09-14 02:54:40,388 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Principal admin::internal synchronized >2014-09-14 02:54:56,928 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-14 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Start deleteAgedOutCommandEntities >2014-09-14 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-72) Start deleteAgedOutAuditLogs >2014-09-14 03:35:35,008 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-52) [396e9d49] Finished deleteAgedOutCommandEntities >2014-09-14 03:35:35,009 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-72) Finished deleteAgedOutAuditLogs >2014-09-14 03:54:40,398 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-64) [435a3a3] Principal admin::internal synchronized >2014-09-14 03:54:56,949 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-09-14 04:54:40,406 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-57) Principal admin::internal synchronized >2014-09-14 04:54:56,968 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-99) Attempting to update VMs/Templates Ovf. >2014-09-14 05:54:40,415 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-92) [13398f04] Principal admin::internal synchronized >2014-09-14 05:54:56,996 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-09-14 06:54:40,426 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-72) Principal admin::internal synchronized >2014-09-14 06:54:57,019 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-09-14 07:54:40,436 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-67) Principal admin::internal synchronized >2014-09-14 07:54:57,043 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-09-14 08:54:40,446 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-64) [435a3a3] Principal admin::internal synchronized >2014-09-14 08:54:57,066 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-84) Attempting to update VMs/Templates Ovf. >2014-09-14 09:54:40,455 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) [18c02297] Principal admin::internal synchronized >2014-09-14 09:54:57,088 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-100) Attempting to update VMs/Templates Ovf. >2014-09-14 10:54:40,464 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-14 10:54:57,116 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-09-14 11:54:40,475 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-14 11:54:57,138 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2014-09-14 12:54:40,489 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-14 12:54:57,157 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2014-09-14 13:54:40,498 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-14 13:54:57,181 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update VMs/Templates Ovf. >2014-09-14 14:54:40,508 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-14) Principal admin::internal synchronized >2014-09-14 14:54:57,200 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) Attempting to update VMs/Templates Ovf. >2014-09-14 15:54:40,520 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-14 15:54:57,222 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-14 16:54:40,530 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-24) Principal admin::internal synchronized >2014-09-14 16:54:57,243 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-89) Attempting to update VMs/Templates Ovf. >2014-09-14 17:54:40,537 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-92) [13398f04] Principal admin::internal synchronized >2014-09-14 17:54:57,267 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-14 18:54:40,549 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-14 18:54:57,289 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-09-14 19:54:40,560 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-21) Principal admin::internal synchronized >2014-09-14 19:54:57,315 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-79) Attempting to update VMs/Templates Ovf. >2014-09-14 20:54:40,569 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) Principal admin::internal synchronized >2014-09-14 20:54:57,337 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-09-14 21:54:40,578 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-14 21:54:57,355 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-77) Attempting to update VMs/Templates Ovf. >2014-09-14 22:54:40,588 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-14 22:54:57,377 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-69) [2589e2b6] Attempting to update VMs/Templates Ovf. >2014-09-14 23:54:40,598 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-09-14 23:54:57,399 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-15 00:54:40,610 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-25) Principal admin::internal synchronized >2014-09-15 00:54:57,420 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-15 01:54:40,621 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) Principal admin::internal synchronized >2014-09-15 01:54:57,443 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-09-15 02:54:40,635 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-85) Principal admin::internal synchronized >2014-09-15 02:54:57,463 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2014-09-15 03:35:35,000 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Start deleteAgedOutAuditLogs >2014-09-15 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Start deleteAgedOutCommandEntities >2014-09-15 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Finished deleteAgedOutAuditLogs >2014-09-15 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-34) Finished deleteAgedOutCommandEntities >2014-09-15 03:54:40,644 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-15 03:54:57,484 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) Attempting to update VMs/Templates Ovf. >2014-09-15 04:54:40,653 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-65) Principal admin::internal synchronized >2014-09-15 04:54:57,508 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2014-09-15 05:54:40,663 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-20) [3de201f6] Principal admin::internal synchronized >2014-09-15 05:54:57,530 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-15 06:54:40,672 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-09-15 06:54:57,552 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-15 07:54:40,684 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-47) Principal admin::internal synchronized >2014-09-15 07:54:57,572 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-15 08:54:40,695 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-72) Principal admin::internal synchronized >2014-09-15 08:54:57,593 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-09-15 09:02:42,979 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-10) Running command: LoginAdminUserCommand internal: false. >2014-09-15 09:02:42,991 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-10) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-15 09:03:15,143 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-09-15 09:03:15,157 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-15 09:08:43,452 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-15 09:08:43,452 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-15 09:54:40,705 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-3) Principal admin::internal synchronized >2014-09-15 09:54:57,614 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) [396e9d49] Attempting to update VMs/Templates Ovf. >2014-09-15 10:38:25,224 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-15 10:38:25,224 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-15 10:38:28,246 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-15 10:38:28,246 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-15 10:38:46,458 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-4) [570ceea3] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-15 10:38:46,505 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-15 10:38:46,509 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] START, SetVdsStatusVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 2402e8f3 >2014-09-15 10:38:46,509 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] VDS rack is spm and moved from up calling resetIrs. >2014-09-15 10:38:46,511 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = f689c28a-07de-4408-ac86-81382b6dd288, ignoreStopFailed = false), log id: 7575b7c7 >2014-09-15 10:38:46,516 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] START, SpmStopVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 7a64c4f7 >2014-09-15 10:38:46,521 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-15 10:38:47,744 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] FINISH, SpmStopVDSCommand, log id: 7a64c4f7 >2014-09-15 10:38:47,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] FINISH, ResetIrsVDSCommand, log id: 7575b7c7 >2014-09-15 10:38:47,780 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] FINISH, SetVdsStatusVDSCommand, log id: 2402e8f3 >2014-09-15 10:38:47,830 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-15 10:38:47,870 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-46) [570ceea3] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDS >2014-09-15 10:38:47,883 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-46) [570ceea3] Correlation ID: 570ceea3, Job ID: 48728116-b278-4d4b-ae54-650d53338936, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-15 10:38:48,415 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = f689c28a-07de-4408-ac86-81382b6dd288 : rack >2014-09-15 10:38:48,468 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-13) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-15 10:38:48,468 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-13) Removing vds [f689c28a-07de-4408-ac86-81382b6dd288] from the domain in maintenance cache >2014-09-15 10:38:48,469 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-13) Removing host(s) [f689c28a-07de-4408-ac86-81382b6dd288] from hosts unseen domain report cache >2014-09-15 10:38:48,474 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-22) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 645696c4 >2014-09-15 10:38:51,265 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-22) FINISH, DisconnectStoragePoolVDSCommand, log id: 645696c4 >2014-09-15 10:38:51,267 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-22) [191d6f35] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-15 10:38:51,282 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-22) [191d6f35] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = f689c28a-07de-4408-ac86-81382b6dd288, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2e089718 >2014-09-15 10:38:51,512 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-22) [191d6f35] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 2e089718 >2014-09-15 10:38:57,773 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-89) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-15 10:38:57,777 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-89) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-15 10:38:57,785 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-89) [4a5fa6ef] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-15 10:38:57,804 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) [4a5fa6ef] Correlation ID: 4a5fa6ef, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-15 10:39:03,134 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-1) [155bd566] Lock Acquired to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-15 10:39:03,149 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-45) [155bd566] Running command: RemoveVdsCommand internal: false. Entities affected : ID: f689c28a-07de-4408-ac86-81382b6dd288 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-15 10:39:03,209 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-45) [155bd566] START, RemoveVdsVDSCommand( HostId = f689c28a-07de-4408-ac86-81382b6dd288), log id: 6b8dc02c >2014-09-15 10:39:03,209 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-45) [155bd566] vdsManager::disposing >2014-09-15 10:39:03,210 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-45) [155bd566] FINISH, RemoveVdsVDSCommand, log id: 6b8dc02c >2014-09-15 10:39:03,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-45) [155bd566] Correlation ID: 155bd566, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-15 10:39:03,218 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-45) [155bd566] Lock freed to object EngineLock [exclusiveLocks= key: f689c28a-07de-4408-ac86-81382b6dd288 value: VDS >, sharedLocks= ] >2014-09-15 10:54:40,715 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-20) [3de201f6] Principal admin::internal synchronized >2014-09-15 10:54:57,635 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) [18c02297] Attempting to update VMs/Templates Ovf. >2014-09-15 11:54:40,724 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-15 11:54:57,638 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VMs/Templates Ovf. >2014-09-15 12:54:40,733 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-15 12:54:57,640 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-09-15 13:21:08,610 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [27af7dec] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' >2014-09-15 13:21:08,634 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-3) [27af7dec] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-15 13:21:08,726 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 13:21:08,741 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-15 13:21:08,744 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] START, RemoveVdsVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649), log id: 161c6ca7 >2014-09-15 13:21:08,744 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] FINISH, RemoveVdsVDSCommand, log id: 161c6ca7 >2014-09-15 13:21:08,746 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] START, AddVdsVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649), log id: 27f8ceed >2014-09-15 13:21:08,746 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] AddVds - entered , starting logic to add VDS cbaace0a-0b00-4c78-89b8-efc33fab4649 >2014-09-15 13:21:08,748 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] AddVds - VDS cbaace0a-0b00-4c78-89b8-efc33fab4649 was added, will try to add it to the resource manager >2014-09-15 13:21:08,748 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Entered VdsManager constructor >2014-09-15 13:21:08,766 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-15 13:21:08,767 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-3) [4b92f4e5] VDS cbaace0a-0b00-4c78-89b8-efc33fab4649 was added to the Resource Manager >2014-09-15 13:21:08,767 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-3) [4b92f4e5] FINISH, AddVdsVDSCommand, log id: 27f8ceed >2014-09-15 13:21:08,783 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-15 13:21:08,817 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 13:21:08,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [4b92f4e5] Correlation ID: 27af7dec, Job ID: 1bf3f884-4bb0-4e49-8da4-116fbd278522, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-15 13:21:08,838 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 13:21:08,839 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] Before Installation host cbaace0a-0b00-4c78-89b8-efc33fab4649, rack >2014-09-15 13:21:08,844 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7e8528aa >2014-09-15 13:21:08,858 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] FINISH, SetVdsStatusVDSCommand, log id: 7e8528aa >2014-09-15 13:21:08,914 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-14) [58689921] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e >2014-09-15 13:21:08,927 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [58689921] Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e. >2014-09-15 13:21:08,999 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-14) [58689921] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-15 13:21:09,010 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-14) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-15 13:21:11,771 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-45) Initializing Host: rack >2014-09-15 13:21:20,130 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-15 13:21:20,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-15 13:21:20,143 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libselinux-python-2.0.94-5.8.el6.x86_64 >2014-09-15 13:21:20,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: libselinux-python-2.0.94-5.8.el6.x86_64. >2014-09-15 13:21:20,156 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-15 13:21:20,164 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-15 13:21:20,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-15 13:21:20,174 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-15 13:21:20,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-15 13:21:20,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-15 13:21:20,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 1/1: libselinux-python-2.0.94-5.8.el6.x86_64 >2014-09-15 13:21:20,767 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 1/1: libselinux-python-2.0.94-5.8.el6.x86_64. >2014-09-15 13:21:24,471 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.8.el6 - u >2014-09-15 13:21:24,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/1: libselinux-python.x86_64 0:2.0.94-5.8.el6 - u. >2014-09-15 13:21:25,399 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-15 13:21:25,408 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-15 13:21:25,426 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-15 13:21:25,436 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-15 13:21:25,445 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-15 13:21:25,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-15 13:21:28,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-15 13:21:28,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-15 13:21:28,987 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-15 13:21:28,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-15 13:21:29,371 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-15 13:21:29,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-15 13:21:29,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140915132109-q2yfho.log' >2014-09-15 13:21:29,437 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140915132109-q2yfho.log'. >2014-09-15 13:21:29,536 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-15 13:21:29,566 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 13:21:29,690 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-15 13:21:29,700 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-15 13:21:29,728 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-15 13:21:29,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-15 13:21:29,784 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-15 13:21:29,796 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-15 13:21:30,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-15 13:21:30,022 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-15 13:21:30,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-15 13:21:30,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-15 13:21:30,110 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-15 13:21:30,121 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-15 13:21:59,876 ERROR [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum [u'libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 requires libvirt = 0.10.2-29.el6_5.12'] >2014-09-15 13:21:59,888 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Failed to install Host rack. Yum [u'libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 requires libvirt = 0.10.2-29.el6_5.12']. >2014-09-15 13:21:59,888 ERROR [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Failed to execute stage 'Package installation': [u'libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 requires libvirt = 0.10.2-29.el6_5.12'] >2014-09-15 13:21:59,898 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Failed to install Host rack. Failed to execute stage 'Package installation': [u'libvirt-lock-sanlock-0.10.2-29.el6_5.12.x86_64 requires libvirt = 0.10.2-29.el6_5.12']. >2014-09-15 13:21:59,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Performing yum transaction rollback >2014-09-15 13:21:59,906 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Performing yum transaction rollback. >2014-09-15 13:21:59,913 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-15 13:21:59,925 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-15 13:21:59,946 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140915132159-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-58689921.log' >2014-09-15 13:21:59,956 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140915132159-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-58689921.log'. >2014-09-15 13:22:00,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-15 13:22:00,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 58689921, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-15 13:22:00,263 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-14) SSH error running command root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True': java.io.IOException: Command returned failure code 1 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.executeCommand(SSHClient.java:527) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:318) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1118) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 13:22:00,267 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-14) [58689921] Error during host dell-r210ii-16.rhev.lab.eng.brq.redhat.com install: java.io.IOException: Command returned failure code 1 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.executeCommand(SSHClient.java:527) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:318) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1118) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 13:22:00,275 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] Host installation failed for host cbaace0a-0b00-4c78-89b8-efc33fab4649, rack.: java.io.IOException: Command returned failure code 1 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' > at org.ovirt.engine.core.uutils.ssh.SSHClient.executeCommand(SSHClient.java:527) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.executeCommand(SSHDialog.java:318) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1118) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2249) [bll.jar:] > at org.ovirt.engine.core.bll.AddVdsCommand$3.run(AddVdsCommand.java:195) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 13:22:00,281 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=InstallFailed, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 300b54bb >2014-09-15 13:22:00,301 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] FINISH, SetVdsStatusVDSCommand, log id: 300b54bb >2014-09-15 13:22:00,312 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [58689921] Correlation ID: 58689921, Job ID: 1bf3f884-4bb0-4e49-8da4-116fbd278522, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. Command returned failure code 1 during SSH session 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com'. >2014-09-15 13:22:00,321 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-14) [58689921] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 13:25:18,928 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-9) [66f7dd23] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 13:25:18,968 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Running command: ActivateVdsCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-15 13:25:18,968 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-15 13:25:18,968 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-15 13:25:18,975 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 575e520d >2014-09-15 13:25:18,991 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] FINISH, SetVdsStatusVDSCommand, log id: 575e520d >2014-09-15 13:25:19,048 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-15 13:25:19,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Correlation ID: 66f7dd23, Job ID: dae3a3ba-32b9-4329-b50b-b7f3359d93ff, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-15 13:25:19,067 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-21) [66f7dd23] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 13:25:21,086 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:21,088 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:21,089 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:21,090 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-6) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:21,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-6) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:21,097 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:21,111 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-6) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:21,128 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-6) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:21,133 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Failed to refresh VDS , vds = cbaace0a-0b00-4c78-89b8-efc33fab4649 : rack, VDS Network Error, continuing. >java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:24,141 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:24,142 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:24,143 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:24,143 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-1) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:24,146 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-1) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:24,147 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:24,153 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:24,161 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-1) Failed to refresh VDS , vds = cbaace0a-0b00-4c78-89b8-efc33fab4649 : rack, VDS Network Error, continuing. >java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:27,168 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:27,169 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:27,170 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:27,170 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-67) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:27,173 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-67) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:27,173 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Host rack is not responding. It will stay in Connecting state for a grace period of 60 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:27,232 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-67) Server failed to respond, vds_id = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 60, error = java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:27,245 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-15 13:25:27,250 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-10) ResourceManager::vdsNotResponding entered for Host cbaace0a-0b00-4c78-89b8-efc33fab4649, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 13:25:27,297 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:27,326 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-10) [4f21d992] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 13:25:27,329 INFO [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-10) [4f21d992] Opening SSH Soft Fencing session on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 13:25:27,491 ERROR [org.ovirt.engine.core.bll.SshSoftFencingCommand] (org.ovirt.thread.pool-6-thread-10) [4f21d992] SSH Soft Fencing command failed on host dell-r210ii-16.rhev.lab.eng.brq.redhat.com: SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host >Stdout: >Stderr: >Stacktrace: javax.naming.AuthenticationException: SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host: javax.naming.AuthenticationException: SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.uutils.ssh.SSHClient.authenticate(SSHClient.java:365) [uutils.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeSshSoftFencingCommand(SshSoftFencingCommand.java:88) [bll.jar:] > at org.ovirt.engine.core.bll.SshSoftFencingCommand.executeCommand(SshSoftFencingCommand.java:53) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:430) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:411) [bll.jar:] > at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:616) [bll.jar:] > at sun.reflect.GeneratedMethodAccessor152.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:74) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:84) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:97) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:93) [jboss-as-weld.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:52) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:259) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:398) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:242) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:185) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation.jar:1.1.2.Final-redhat-1] > at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) [jboss-as-ee.jar:7.4.0.Final-redhat-19] > at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view7.runInternalAction(Unknown Source) [bll.jar:] > at org.ovirt.engine.core.bll.VdsEventListener$4.run(VdsEventListener.java:243) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 13:25:27,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-10) [4f21d992] Correlation ID: 4f21d992, Job ID: 4d9e3299-5058-4104-be67-ec69b96f92bf, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-15 13:25:27,593 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-10) [17871262] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS_FENCE >, sharedLocks= ] >2014-09-15 13:25:27,593 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-10) [17871262] Failed to run Fence script on vds:rack. >2014-09-15 13:25:27,608 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-10) [17871262] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-09-15 13:25:27,609 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-10) [17871262] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-09-15 13:25:27,609 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-10) [17871262] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS_FENCE >, sharedLocks= ] >2014-09-15 13:25:30,320 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:30,320 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:30,321 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:30,322 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-13) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:30,324 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-13) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:30,325 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-13) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:30,338 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-13) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:33,351 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:33,352 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:33,352 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:33,353 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-99) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:33,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-99) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:33,357 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-99) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:33,369 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-99) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:36,381 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:36,382 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:36,383 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:36,383 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-36) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:36,386 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-36) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:36,386 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-36) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:36,404 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-36) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:39,417 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:39,418 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:39,418 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:39,419 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-25) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:39,421 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-25) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:39,421 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-25) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:39,433 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-25) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:42,446 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:42,447 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:42,447 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:42,448 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-20) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:42,451 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-20) [3de201f6] Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:42,452 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-20) [3de201f6] Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:42,465 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-20) [3de201f6] Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:45,478 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:45,479 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:45,479 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:45,480 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-50) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:45,483 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-50) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:45,483 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-50) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:45,499 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-50) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:48,511 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:48,512 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:48,512 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:48,513 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-48) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:48,516 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-48) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:48,516 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-48) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:48,530 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:51,547 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:51,548 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:51,549 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:51,549 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-51) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:51,552 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-51) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:51,553 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-51) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:51,568 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-51) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:54,583 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:54,584 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:54,584 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:54,585 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-75) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:54,587 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-75) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:54,587 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-75) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:54,598 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-75) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:57,610 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:25:57,611 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:57,611 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:57,612 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-74) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:25:57,614 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-74) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:25:57,614 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-74) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:25:57,627 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-74) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:26:00,640 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 13:26:00,641 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:26:00,642 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:26:00,642 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-5) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:26:00,645 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-5) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 13:26:00,646 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-5) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 13:26:00,659 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-5) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 13:26:01,330 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-17) [24d966c0] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-15 13:26:01,430 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-17) [24d966c0] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 13:26:01,437 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-17) [24d966c0] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 2fa6f55f >2014-09-15 13:26:01,467 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-17) [24d966c0] FINISH, SetVdsStatusVDSCommand, log id: 2fa6f55f >2014-09-15 13:26:01,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-17) [24d966c0] Correlation ID: 24d966c0, Job ID: ca405ebb-e91c-4ee2-961b-5992e85902e9, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-15 13:54:40,743 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-64) [435a3a3] Principal admin::internal synchronized >2014-09-15 13:54:57,643 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-09-15 14:54:40,754 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-32) Principal admin::internal synchronized >2014-09-15 14:54:57,646 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-78) Attempting to update VMs/Templates Ovf. >2014-09-15 14:56:24,315 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-15 14:56:24,315 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-15 14:56:24,315 INFO [org.ovirt.engine.core.bll.InstallVdsCommand] (ajp-/127.0.0.1:8702-1) [12ad9466] Running command: InstallVdsCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN >2014-09-15 14:56:24,338 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-15 14:56:24,338 WARN [org.ovirt.engine.core.compat.backendcompat.PropertyInfo] (ajp-/127.0.0.1:8702-1) Unable to get value of property: vdsName for class org.ovirt.engine.core.common.businessentities.VdsStatic >2014-09-15 14:56:24,341 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (ajp-/127.0.0.1:8702-1) [12ad9466] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:56:24,345 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 14:56:24,346 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Before Installation host cbaace0a-0b00-4c78-89b8-efc33fab4649, rack >2014-09-15 14:56:24,361 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [12ad9466] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-15 14:56:24,369 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7fa77acf >2014-09-15 14:56:24,397 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] FINISH, SetVdsStatusVDSCommand, log id: 7fa77acf >2014-09-15 14:56:24,405 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [12ad9466] Correlation ID: 12ad9466, Call Stack: null, Custom Event ID: -1, Message: Host rack configuration was updated by admin. >2014-09-15 14:56:24,487 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e >2014-09-15 14:56:24,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Correlation ID: 12ad9466, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e. >2014-09-15 14:56:24,563 ERROR [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Error during host dell-r210ii-16.rhev.lab.eng.brq.redhat.com install: javax.naming.AuthenticationException: SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.uutils.ssh.SSHClient.authenticate(SSHClient.java:365) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.authenticate(SSHDialog.java:270) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1100) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:191) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.runCommands(MultipleActionsRunner.java:158) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$2.run(MultipleActionsRunner.java:167) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 14:56:24,569 ERROR [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Host installation failed for host cbaace0a-0b00-4c78-89b8-efc33fab4649, rack.: javax.naming.AuthenticationException: SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host > at org.ovirt.engine.core.uutils.ssh.SSHClient.authenticate(SSHClient.java:365) [uutils.jar:] > at org.ovirt.engine.core.uutils.ssh.SSHDialog.authenticate(SSHDialog.java:270) [uutils.jar:] > at org.ovirt.engine.core.bll.VdsDeploy.execute(VdsDeploy.java:1100) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.installHost(InstallVdsInternalCommand.java:151) [bll.jar:] > at org.ovirt.engine.core.bll.InstallVdsInternalCommand.executeCommand(InstallVdsInternalCommand.java:78) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1167) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1298) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1920) [bll.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:174) [utils.jar:] > at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:116) [utils.jar:] > at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1319) [bll.jar:] > at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:344) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.executeValidatedCommand(MultipleActionsRunner.java:191) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner.runCommands(MultipleActionsRunner.java:158) [bll.jar:] > at org.ovirt.engine.core.bll.MultipleActionsRunner$2.run(MultipleActionsRunner.java:167) [bll.jar:] > at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalWrapperRunnable.run(ThreadPoolUtil.java:90) [utils.jar:] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] > >2014-09-15 14:56:24,573 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=InstallFailed, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 513cc0d4 >2014-09-15 14:56:24,588 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] FINISH, SetVdsStatusVDSCommand, log id: 513cc0d4 >2014-09-15 14:56:24,599 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Correlation ID: 12ad9466, Call Stack: null, Custom Event ID: -1, Message: Host rack installation failed. SSH authentication to 'root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com' failed. Please verify provided credentials. Make sure key is authorized at host. >2014-09-15 14:56:24,599 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-9) [12ad9466] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:56:34,942 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-1) [df36e6b] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_OPERATIONAL >2014-09-15 14:56:45,976 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-7) [28d25be] CanDoAction of action MaintenanceNumberOfVdss failed. Reasons:VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_VDS_IS_NOT_OPERATIONAL >2014-09-15 14:56:53,376 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-9) [34d14d29] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:56:53,415 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Running command: ActivateVdsCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-15 14:56:53,416 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-15 14:56:53,416 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-15 14:56:53,423 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6d4dda15 >2014-09-15 14:56:53,445 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] FINISH, SetVdsStatusVDSCommand, log id: 6d4dda15 >2014-09-15 14:56:53,454 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-15 14:56:53,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Correlation ID: 34d14d29, Job ID: ee957b99-35d3-431a-aa3d-8ed134edcd92, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-15 14:56:53,469 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-16) [34d14d29] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:56:55,317 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:56:55,318 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:55,318 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:55,319 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-80) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:55,321 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-80) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 14:56:55,322 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 14:56:55,334 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 14:56:55,401 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-80) Server failed to respond, vds_id = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds_name = rack, vm_count = 0, spm_status = None, non-responsive_timeout (seconds) = 120, error = java.net.NoRouteToHostException: No route to host >2014-09-15 14:56:55,419 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-80) Correlation ID: null, Call Stack: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) > at sun.nio.ch.Net.connect(Net.java:465) > at sun.nio.ch.Net.connect(Net.java:457) > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) >, Custom Event ID: -1, Message: Host rack is non responsive. >2014-09-15 14:56:55,423 INFO [org.ovirt.engine.core.bll.VdsEventListener] (org.ovirt.thread.pool-6-thread-44) ResourceManager::vdsNotResponding entered for Host cbaace0a-0b00-4c78-89b8-efc33fab4649, dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 14:56:55,455 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-80) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:55,461 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-44) [e69e55e] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS_FENCE >, sharedLocks= ] >2014-09-15 14:56:55,461 ERROR [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-44) [e69e55e] Failed to run Fence script on vds:rack. >2014-09-15 14:56:55,471 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-44) [e69e55e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Host rack became non responsive. It has no power management configured. Please check the host status, manually reboot it, and click "Confirm Host Has Been Rebooted" >2014-09-15 14:56:55,471 WARN [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-44) [e69e55e] CanDoAction of action VdsNotRespondingTreatment failed. Reasons:VAR__ACTION__RESTART,VDS_FENCE_DISABLED >2014-09-15 14:56:55,473 INFO [org.ovirt.engine.core.bll.VdsNotRespondingTreatmentCommand] (org.ovirt.thread.pool-6-thread-44) [e69e55e] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS_FENCE >, sharedLocks= ] >2014-09-15 14:56:58,468 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:56:58,469 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:58,469 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:58,469 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (DefaultQuartzScheduler_Worker-100) Exception during connection: java.util.concurrent.ExecutionException: java.net.NoRouteToHostException: No route to host > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer.getCapabilities(JsonRpcVdsServer.java:184) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand.executeVdsBrokerCommand(GetCapabilitiesVDSCommand.java:15) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:96) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:58,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler_Worker-100) Command GetCapabilitiesVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, vds=Host[rack,cbaace0a-0b00-4c78-89b8-efc33fab4649]) execution failed. Exception: VDSNetworkException: java.net.NoRouteToHostException: No route to host >2014-09-15 14:56:58,472 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-100) Host rack is not responding. It will stay in Connecting state for a grace period of 120 seconds and after that an attempt to fence the host will be issued. >2014-09-15 14:56:58,484 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.net.NoRouteToHostException: No route to host > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createNetworkException(VdsBrokerCommand.java:126) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:101) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.refreshCapabilities(VdsManager.java:567) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refreshVdsRunTimeInfo(VdsUpdateRunTimeInfo.java:639) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo.refresh(VdsUpdateRunTimeInfo.java:486) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VdsManager.onTimer(VdsManager.java:236) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor56.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] >Caused by: java.net.NoRouteToHostException: No route to host > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:56:59,951 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-27) [45aa12bf] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-15 14:56:59,992 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-27) [45aa12bf] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDS >2014-09-15 14:56:59,997 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-27) [45aa12bf] START, SetVdsStatusVDSCommand(HostName = rack, HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649, status=Maintenance, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 504594e8 >2014-09-15 14:57:00,019 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-27) [45aa12bf] FINISH, SetVdsStatusVDSCommand, log id: 504594e8 >2014-09-15 14:57:00,027 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) [45aa12bf] Correlation ID: 45aa12bf, Job ID: 46c9706c-e942-4dec-be1d-de5b5fd1da64, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-15 14:57:04,718 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-6) [178bf077] Lock Acquired to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:57:04,728 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-36) [178bf077] Running command: RemoveVdsCommand internal: false. Entities affected : ID: cbaace0a-0b00-4c78-89b8-efc33fab4649 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-15 14:57:04,753 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-36) [178bf077] START, RemoveVdsVDSCommand( HostId = cbaace0a-0b00-4c78-89b8-efc33fab4649), log id: 2ede9daa >2014-09-15 14:57:04,753 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-36) [178bf077] vdsManager::disposing >2014-09-15 14:57:04,754 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-36) [178bf077] FINISH, RemoveVdsVDSCommand, log id: 2ede9daa >2014-09-15 14:57:04,762 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-36) [178bf077] Correlation ID: 178bf077, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-15 14:57:04,762 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-36) [178bf077] Lock freed to object EngineLock [exclusiveLocks= key: cbaace0a-0b00-4c78-89b8-efc33fab4649 value: VDS >, sharedLocks= ] >2014-09-15 14:57:28,899 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [1714916] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-15 14:57:28,953 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-15 14:57:28,965 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-15 14:57:28,968 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] START, RemoveVdsVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 4e1937bb >2014-09-15 14:57:28,968 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] FINISH, RemoveVdsVDSCommand, log id: 4e1937bb >2014-09-15 14:57:28,970 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] START, AddVdsVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 67c8cd03 >2014-09-15 14:57:28,970 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] AddVds - entered , starting logic to add VDS c1aae4b4-6d89-47af-930c-d8b9c6e8323b >2014-09-15 14:57:28,972 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] AddVds - VDS c1aae4b4-6d89-47af-930c-d8b9c6e8323b was added, will try to add it to the resource manager >2014-09-15 14:57:28,972 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [366bf2d2] Entered VdsManager constructor >2014-09-15 14:57:28,988 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [366bf2d2] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-15 14:57:28,989 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-6) [366bf2d2] VDS c1aae4b4-6d89-47af-930c-d8b9c6e8323b was added to the Resource Manager >2014-09-15 14:57:28,989 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [366bf2d2] FINISH, AddVdsVDSCommand, log id: 67c8cd03 >2014-09-15 14:57:29,003 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [366bf2d2] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-15 14:57:29,029 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] Lock Acquired to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-15 14:57:29,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [366bf2d2] Correlation ID: 1714916, Job ID: 1f350336-37de-4e08-ae91-5f3a0b97272c, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-15 14:57:29,047 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-15 14:57:29,047 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] Before Installation host c1aae4b4-6d89-47af-930c-d8b9c6e8323b, rack >2014-09-15 14:57:29,051 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6cf66c39 >2014-09-15 14:57:29,059 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] FINISH, SetVdsStatusVDSCommand, log id: 6cf66c39 >2014-09-15 14:57:29,121 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-26) [242c818e] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e >2014-09-15 14:57:29,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-26) [242c818e] Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e. >2014-09-15 14:57:29,203 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-26) [242c818e] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-15 14:57:29,214 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-26) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-15 14:57:30,270 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-15 14:57:30,279 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-15 14:57:30,295 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-15 14:57:30,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-15 14:57:30,305 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-15 14:57:30,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-15 14:57:31,992 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Initializing Host: rack >2014-09-15 14:57:35,872 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-15 14:57:35,895 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-15 14:57:35,895 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-15 14:57:35,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-15 14:57:36,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-15 14:57:36,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-15 14:57:36,288 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140915145729-rowic8.log' >2014-09-15 14:57:36,298 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140915145729-rowic8.log'. >2014-09-15 14:57:36,391 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-15 14:57:36,426 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 14:57:36,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-15 14:57:36,535 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-15 14:57:36,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-15 14:57:36,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-15 14:57:36,613 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-15 14:57:36,624 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-15 14:57:36,836 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-15 14:57:36,846 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-15 14:57:36,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-15 14:57:36,914 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-15 14:57:36,915 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-15 14:57:36,923 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-15 14:57:39,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-15 14:57:39,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-15 14:57:40,846 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64 >2014-09-15 14:57:40,856 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64. >2014-09-15 14:57:40,856 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-15 14:57:40,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-15 14:57:40,866 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-15 14:57:40,878 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-15 14:57:40,920 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-15 14:57:40,934 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-15 14:57:41,117 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 1/1: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64 >2014-09-15 14:57:41,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 1/1: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64. >2014-09-15 14:57:44,907 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/1: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.446.el6 - u >2014-09-15 14:57:44,917 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/1: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.446.el6 - u. >2014-09-15 14:57:45,112 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-15 14:57:45,121 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-15 14:57:46,392 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-15 14:57:46,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-15 14:57:47,648 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-15 14:57:47,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-15 14:57:47,776 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-15 14:57:47,785 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-15 14:57:48,780 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-15 14:57:48,790 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-15 14:57:48,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-15 14:57:48,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-15 14:57:52,288 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-15 14:57:52,305 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-15 14:57:52,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140915145752-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-242c818e.log' >2014-09-15 14:57:52,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140915145752-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-242c818e.log'. >2014-09-15 14:57:52,592 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-15 14:57:52,604 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 242c818e, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-15 14:57:52,720 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:57:52,720 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:57:52,721 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:57:52,721 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-6-thread-31) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:918) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:914) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:405) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:394) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-15 14:57:52,723 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-15 14:57:52,726 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-26) [242c818e] Command PollVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-15 14:57:53,227 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:57:53,348 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-26) [242c818e] Engine managed to communicate with VDSM agent on host rack >2014-09-15 14:57:53,845 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-26) [11559cf5] Running command: SetupNetworksCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-15 14:57:53,848 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-26) [11559cf5] START, SetupNetworksVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=e701d56d-e183-475e-9cd1-f450949daed5, vdsId=c1aae4b4-6d89-47af-930c-d8b9c6e8323b, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=83ba50f0-2ef0-44c3-a58a-fbbc9e1cee6d, vdsId=c1aae4b4-6d89-47af-930c-d8b9c6e8323b, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 13a1fba8 >2014-09-15 14:57:53,851 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-26) [11559cf5] FINISH, SetupNetworksVDSCommand, log id: 13a1fba8 >2014-09-15 14:57:53,851 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:58:09,499 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:58:09,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-26) [11559cf5] Correlation ID: 11559cf5, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-15 14:58:09,675 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-15 14:58:09,677 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] START, SetSafeNetworkConfigVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 72c18943 >2014-09-15 14:58:09,691 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] FINISH, SetSafeNetworkConfigVDSCommand, log id: 72c18943 >2014-09-15 14:58:09,706 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] Correlation ID: 61e77b7e, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host rack >2014-09-15 14:58:09,709 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6266080d >2014-09-15 14:58:09,728 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] FINISH, SetVdsStatusVDSCommand, log id: 6266080d >2014-09-15 14:58:09,728 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] After Installation host rack, VDS >2014-09-15 14:58:09,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] Correlation ID: 242c818e, Job ID: 1f350336-37de-4e08-ae91-5f3a0b97272c, Call Stack: null, Custom Event ID: -1, Message: Host rack installed >2014-09-15 14:58:09,754 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-26) [61e77b7e] Lock freed to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-15 14:58:11,087 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vds=Host[rack,c1aae4b4-6d89-47af-930c-d8b9c6e8323b]), log id: 3e184278 >2014-09-15 14:58:11,139 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) FINISH, GetHardwareInfoVDSCommand, log id: 3e184278 >2014-09-15 14:58:11,198 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-59) [776ded53] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-15 14:58:11,243 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) [776ded53] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vds=Host[rack,c1aae4b4-6d89-47af-930c-d8b9c6e8323b]), log id: 3975edb7 >2014-09-15 14:58:11,248 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-59) [776ded53] FINISH, GetHardwareInfoVDSCommand, log id: 3975edb7 >2014-09-15 14:58:11,298 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-59) [56176875] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-15 14:58:11,315 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-15 14:58:11,328 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 53951449 >2014-09-15 14:58:11,882 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 53951449 >2014-09-15 14:58:11,883 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] Host rack storage connection was succeeded >2014-09-15 14:58:11,895 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-24) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 3b6f58b8 >2014-09-15 14:58:12,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-24) FINISH, ConnectStoragePoolVDSCommand, log id: 3b6f58b8 >2014-09-15 14:58:12,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: d80bda5 >2014-09-15 14:58:12,957 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-59) [11bc3115] FINISH, SetMOMPolicyParametersVDSCommand, log id: d80bda5 >2014-09-15 14:58:12,973 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [11bc3115] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-15 14:58:12,987 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [11bc3115] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-15 14:58:12,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [11bc3115] Correlation ID: 56176875, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-15 14:58:13,029 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-59) [714853c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-15 14:58:13,038 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-59) [384f2971] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-15 14:58:17,582 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-94) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-15 14:58:17,601 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-94) starting spm on vds rack, storage pool Default, prevId -1, LVER 0 >2014-09-15 14:58:17,604 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-94) START, SpmStartVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=0, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 16500e55 >2014-09-15 14:58:17,614 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-94) spmStart polling started: taskId = fde00e1e-6ea4-4451-afa9-5196d23e25d7 >2014-09-15 14:58:38,726 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-94) spmStart polling ended: taskId = fde00e1e-6ea4-4451-afa9-5196d23e25d7 task status = finished >2014-09-15 14:58:38,730 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-94) spmStart polling ended, spm status: SPM >2014-09-15 14:58:38,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-94) START, HSMClearTaskVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, taskId=fde00e1e-6ea4-4451-afa9-5196d23e25d7), log id: 263a74c2 >2014-09-15 14:58:38,736 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, HSMClearTaskVDSCommand, log id: 263a74c2 >2014-09-15 14:58:38,736 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-94) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@15c2cf07, log id: 16500e55 >2014-09-15 14:58:38,741 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-94) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-15 14:58:38,750 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-15 14:58:38,752 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-15 14:58:38,764 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: f1d0051 >2014-09-15 14:58:38,856 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-43) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-15 14:58:38,858 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 6b20ddd4 >2014-09-15 14:58:38,868 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 6b20ddd4 >2014-09-15 14:58:38,868 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: f1d0051 >2014-09-15 14:58:38,868 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-43) Discovered no tasks on Storage Pool Default >2014-09-15 15:06:24,741 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-15 15:06:24,742 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-15 15:54:40,765 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-49) Principal admin::internal synchronized >2014-09-15 15:54:57,649 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-09-15 16:54:40,774 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-10) Principal admin::internal synchronized >2014-09-15 16:54:57,675 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-15 17:54:40,785 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-15 17:54:57,696 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-92) [13398f04] Attempting to update VMs/Templates Ovf. >2014-09-15 18:54:40,795 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-26) Principal admin::internal synchronized >2014-09-15 18:54:57,721 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2014-09-15 19:54:40,805 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-32) Principal admin::internal synchronized >2014-09-15 19:54:57,743 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VMs/Templates Ovf. >2014-09-15 20:54:40,816 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-15 20:54:57,766 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-27) Attempting to update VMs/Templates Ovf. >2014-09-15 21:54:40,828 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-86) Principal admin::internal synchronized >2014-09-15 21:54:57,797 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-15 22:54:40,837 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-8) Principal admin::internal synchronized >2014-09-15 22:54:57,821 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-10) Attempting to update VMs/Templates Ovf. >2014-09-15 23:54:40,847 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-10) Principal admin::internal synchronized >2014-09-15 23:54:57,842 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-16 00:54:40,857 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-37) Principal admin::internal synchronized >2014-09-16 00:54:57,867 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-55) Attempting to update VMs/Templates Ovf. >2014-09-16 01:54:40,867 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-69) [2589e2b6] Principal admin::internal synchronized >2014-09-16 01:54:57,888 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-16 02:54:40,875 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-45) Principal admin::internal synchronized >2014-09-16 02:54:57,916 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2014-09-16 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-44) Start deleteAgedOutAuditLogs >2014-09-16 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) [435a3a3] Start deleteAgedOutCommandEntities >2014-09-16 03:35:35,006 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-64) [435a3a3] Finished deleteAgedOutCommandEntities >2014-09-16 03:35:35,006 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-44) Finished deleteAgedOutAuditLogs >2014-09-16 03:54:40,888 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-75) Principal admin::internal synchronized >2014-09-16 03:54:57,942 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VMs/Templates Ovf. >2014-09-16 04:54:40,902 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-16 04:54:57,963 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-48) Attempting to update VMs/Templates Ovf. >2014-09-16 05:54:40,918 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-17) Principal admin::internal synchronized >2014-09-16 05:54:57,988 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-21) Attempting to update VMs/Templates Ovf. >2014-09-16 06:54:40,927 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-77) Principal admin::internal synchronized >2014-09-16 06:54:58,015 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-100) Attempting to update VMs/Templates Ovf. >2014-09-16 07:54:40,937 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-16 07:54:58,042 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-16 08:54:40,953 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-60) Principal admin::internal synchronized >2014-09-16 08:54:58,062 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) Attempting to update VMs/Templates Ovf. >2014-09-16 09:30:06,465 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-1) [7ef24a5c] Running command: LogoutUserCommand internal: false. >2014-09-16 09:30:06,482 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [7ef24a5c] Correlation ID: 7ef24a5c, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-16 09:30:06,539 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-16 09:30:06,539 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-16 09:38:43,549 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-16 09:38:43,549 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-16 09:54:40,962 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-99) Principal admin::internal synchronized >2014-09-16 09:54:58,085 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-09-16 10:54:40,974 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-16 10:54:58,107 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VMs/Templates Ovf. >2014-09-16 11:54:40,984 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-47) Principal admin::internal synchronized >2014-09-16 11:54:58,133 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-09-16 12:54:40,994 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-16 12:54:58,156 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-16 12:58:22,843 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-16 12:58:22,843 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-16 12:58:32,714 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-16 12:58:32,714 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-7) Invalid locale found in configuration: >2014-09-16 12:58:38,108 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginAdminUserCommand internal: false. >2014-09-16 12:58:38,119 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-16 12:58:38,171 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-09-16 12:59:05,121 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-7) [7935eb4d] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-16 12:59:05,158 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-16 12:59:05,161 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 29990adf >2014-09-16 12:59:05,161 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] VDS rack is spm and moved from up calling resetIrs. >2014-09-16 12:59:05,161 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, ignoreStopFailed = false), log id: 485d6be0 >2014-09-16 12:59:05,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] START, SpmStopVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 66f2e29c >2014-09-16 12:59:05,168 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-16 12:59:06,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] FINISH, SpmStopVDSCommand, log id: 66f2e29c >2014-09-16 12:59:06,499 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] FINISH, ResetIrsVDSCommand, log id: 485d6be0 >2014-09-16 12:59:06,523 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] FINISH, SetVdsStatusVDSCommand, log id: 29990adf >2014-09-16 12:59:06,573 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-16 12:59:06,623 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-16 12:59:06,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-26) [7935eb4d] Correlation ID: 7935eb4d, Job ID: 205dd5bb-c9ef-4466-b86c-87f91ef3643f, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-16 12:59:08,020 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-8) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = c1aae4b4-6d89-47af-930c-d8b9c6e8323b : rack >2014-09-16 12:59:08,061 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-16 12:59:08,061 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Removing vds [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from the domain in maintenance cache >2014-09-16 12:59:08,062 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Removing host(s) [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from hosts unseen domain report cache >2014-09-16 12:59:08,066 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-8) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 6b0f4def >2014-09-16 12:59:09,682 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-17) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-16 12:59:09,687 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-17) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-16 12:59:09,702 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-17) [2966ac43] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-16 12:59:09,723 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [2966ac43] Correlation ID: 2966ac43, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-16 12:59:10,478 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-8) FINISH, DisconnectStoragePoolVDSCommand, log id: 6b0f4def >2014-09-16 12:59:10,480 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-8) [f4b4848] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-16 12:59:10,496 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-8) [f4b4848] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7ba8fa78 >2014-09-16 12:59:10,789 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-8) [f4b4848] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 7ba8fa78 >2014-09-16 12:59:45,729 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-3) [7e5f1d39] Lock Acquired to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-16 12:59:45,767 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Running command: ActivateVdsCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-16 12:59:45,767 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-16 12:59:45,767 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-16 12:59:45,783 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 6cc0da2a >2014-09-16 12:59:45,800 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] FINISH, SetVdsStatusVDSCommand, log id: 6cc0da2a >2014-09-16 12:59:45,842 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-16 12:59:45,854 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Correlation ID: 7e5f1d39, Job ID: 8b9110ce-f514-4593-b5e3-0895f1b62b73, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-16 12:59:45,865 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7e5f1d39] Lock freed to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-16 12:59:46,915 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-16) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vds=Host[rack,c1aae4b4-6d89-47af-930c-d8b9c6e8323b]), log id: 73f17554 >2014-09-16 12:59:46,926 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-16) FINISH, GetHardwareInfoVDSCommand, log id: 73f17554 >2014-09-16 12:59:46,974 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-16) [5d307f4e] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-16 12:59:46,987 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-16) [2eef1403] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-16 12:59:47,009 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-16 12:59:47,022 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1b69efa6 >2014-09-16 12:59:47,268 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 1b69efa6 >2014-09-16 12:59:47,268 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] Host rack storage connection was succeeded >2014-09-16 12:59:47,279 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 3ae5a21b >2014-09-16 12:59:48,146 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-8) FINISH, ConnectStoragePoolVDSCommand, log id: 3ae5a21b >2014-09-16 12:59:48,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 1529c31f >2014-09-16 12:59:48,218 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-16) [74119e9e] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1529c31f >2014-09-16 12:59:48,231 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [74119e9e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-16 12:59:48,242 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [74119e9e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-16 12:59:48,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [74119e9e] Correlation ID: 2eef1403, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-16 12:59:48,279 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-16) [41ad0916] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-16 12:59:49,839 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-16 12:59:49,848 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) starting spm on vds rack, storage pool Default, prevId -1, LVER 1 >2014-09-16 12:59:49,850 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) START, SpmStartVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 7edcba89 >2014-09-16 12:59:49,858 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling started: taskId = 1cfa3d97-49e8-4d78-8bf5-4e17babc70a9 >2014-09-16 13:00:10,962 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling ended: taskId = 1cfa3d97-49e8-4d78-8bf5-4e17babc70a9 task status = finished >2014-09-16 13:00:10,966 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) spmStart polling ended, spm status: SPM >2014-09-16 13:00:10,967 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-48) START, HSMClearTaskVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, taskId=1cfa3d97-49e8-4d78-8bf5-4e17babc70a9), log id: 1261d651 >2014-09-16 13:00:10,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, HSMClearTaskVDSCommand, log id: 1261d651 >2014-09-16 13:00:10,971 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-48) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@3cc4a7f, log id: 7edcba89 >2014-09-16 13:00:10,974 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-16 13:00:10,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-48) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-16 13:00:10,985 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 60663dbf >2014-09-16 13:00:10,986 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-16 13:00:11,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-16 13:00:11,080 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 2d4cdeb6 >2014-09-16 13:00:11,083 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 2d4cdeb6 >2014-09-16 13:00:11,083 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 60663dbf >2014-09-16 13:00:11,084 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-47) Discovered no tasks on Storage Pool Default >2014-09-16 13:54:41,003 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-33) Principal admin::internal synchronized >2014-09-16 13:54:58,180 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-09-16 14:54:41,012 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-16 14:54:58,202 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-2) Attempting to update VMs/Templates Ovf. >2014-09-16 15:54:41,021 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-74) Principal admin::internal synchronized >2014-09-16 15:54:58,225 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-16 16:54:41,031 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-24) Principal admin::internal synchronized >2014-09-16 16:54:58,248 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-09-16 17:54:13,523 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-16 17:54:13,523 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-16 17:54:17,265 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-16 17:54:17,266 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-16 17:54:23,111 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginAdminUserCommand internal: false. >2014-09-16 17:54:23,127 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-16 17:54:23,463 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-16 17:54:41,041 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-16 17:54:58,273 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-09-16 18:54:41,056 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-48) Principal admin::internal synchronized >2014-09-16 18:54:58,296 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-16 19:54:41,072 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-29) Principal admin::internal synchronized >2014-09-16 19:54:58,319 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-16 20:54:41,081 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-95) Principal admin::internal synchronized >2014-09-16 20:54:58,341 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-09-16 21:54:41,092 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-16 21:54:58,363 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-09-16 22:54:41,103 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-16 22:54:58,385 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-09-16 23:54:41,113 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-48) Principal admin::internal synchronized >2014-09-16 23:54:58,406 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-17 00:54:41,123 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-78) Principal admin::internal synchronized >2014-09-17 00:54:58,426 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-09-17 01:54:41,133 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-22) [191d6f35] Principal admin::internal synchronized >2014-09-17 01:54:58,449 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-45) Attempting to update VMs/Templates Ovf. >2014-09-17 02:54:41,142 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) Principal admin::internal synchronized >2014-09-17 02:54:58,470 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-09-17 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-14) Start deleteAgedOutAuditLogs >2014-09-17 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-14) Finished deleteAgedOutAuditLogs >2014-09-17 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-26) Start deleteAgedOutCommandEntities >2014-09-17 03:35:35,009 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-26) Finished deleteAgedOutCommandEntities >2014-09-17 03:54:41,152 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-17 03:54:58,492 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Attempting to update VMs/Templates Ovf. >2014-09-17 04:54:41,162 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Principal admin::internal synchronized >2014-09-17 04:54:58,512 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-6) Attempting to update VMs/Templates Ovf. >2014-09-17 05:54:41,171 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-17 05:54:58,534 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-21) Attempting to update VMs/Templates Ovf. >2014-09-17 06:54:41,179 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-27) Principal admin::internal synchronized >2014-09-17 06:54:58,556 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-17 07:54:41,190 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-92) [13398f04] Principal admin::internal synchronized >2014-09-17 07:54:58,577 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-95) Attempting to update VMs/Templates Ovf. >2014-09-17 08:54:41,200 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-33) Principal admin::internal synchronized >2014-09-17 08:54:58,598 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-09-17 09:39:08,395 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-17 09:39:08,395 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-6) Invalid locale found in configuration: >2014-09-17 09:54:41,209 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-57) Principal admin::internal synchronized >2014-09-17 09:54:58,616 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-55) Attempting to update VMs/Templates Ovf. >2014-09-17 10:54:41,221 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-74) Principal admin::internal synchronized >2014-09-17 10:54:58,636 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-17 11:02:00,241 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-17 11:02:00,241 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-17 11:02:01,866 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-17 11:02:01,866 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-17 11:02:07,046 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginAdminUserCommand internal: false. >2014-09-17 11:02:07,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-17 11:02:07,125 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginUserCommand internal: false. >2014-09-17 11:02:22,207 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-9) [3248d827] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-17 11:02:22,245 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-17 11:02:22,249 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 188d35a7 >2014-09-17 11:02:22,249 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] VDS rack is spm and moved from up calling resetIrs. >2014-09-17 11:02:22,249 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, ignoreStopFailed = false), log id: 760457b8 >2014-09-17 11:02:22,253 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] START, SpmStopVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 720138e5 >2014-09-17 11:02:22,259 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-17 11:02:23,347 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] FINISH, SpmStopVDSCommand, log id: 720138e5 >2014-09-17 11:02:23,358 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] FINISH, ResetIrsVDSCommand, log id: 760457b8 >2014-09-17 11:02:23,376 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] FINISH, SetVdsStatusVDSCommand, log id: 188d35a7 >2014-09-17 11:02:23,381 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-17 11:02:23,449 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-30) [3248d827] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-17 11:02:23,464 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-30) [3248d827] Correlation ID: 3248d827, Job ID: fef3378e-8746-4496-a472-8dbfca1b1924, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-17 11:02:24,099 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-31) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = c1aae4b4-6d89-47af-930c-d8b9c6e8323b : rack >2014-09-17 11:02:24,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-17 11:02:24,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Removing vds [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from the domain in maintenance cache >2014-09-17 11:02:24,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-7) Removing host(s) [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from hosts unseen domain report cache >2014-09-17 11:02:24,152 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-31) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 27aaeb08 >2014-09-17 11:02:25,010 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-34) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-17 11:02:25,026 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-34) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-17 11:02:25,040 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-34) [2a52126a] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 11:02:25,083 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [2a52126a] Correlation ID: 2a52126a, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-17 11:02:27,067 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-31) FINISH, DisconnectStoragePoolVDSCommand, log id: 27aaeb08 >2014-09-17 11:02:27,069 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-31) [3fa8ffdd] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 11:02:27,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-31) [3fa8ffdd] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 71f06b1d >2014-09-17 11:02:27,334 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-31) [3fa8ffdd] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 71f06b1d >2014-09-17 11:04:30,032 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-10) [7669bf0d] Lock Acquired to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-17 11:04:30,074 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Running command: ActivateVdsCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-17 11:04:30,074 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-17 11:04:30,074 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-17 11:04:30,092 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 137a0d42 >2014-09-17 11:04:30,112 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] FINISH, SetVdsStatusVDSCommand, log id: 137a0d42 >2014-09-17 11:04:30,161 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-17 11:04:30,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Correlation ID: 7669bf0d, Job ID: 40e21782-5a05-4f9e-9eeb-ccd6449a6c4d, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-17 11:04:30,181 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-50) [7669bf0d] Lock freed to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-17 11:04:30,561 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vds=Host[rack,c1aae4b4-6d89-47af-930c-d8b9c6e8323b]), log id: 69e280c3 >2014-09-17 11:04:30,565 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GetHardwareInfoVDSCommand, log id: 69e280c3 >2014-09-17 11:04:30,606 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-47) [21171f51] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-17 11:04:30,616 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-47) [7baa3cbd] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 11:04:30,627 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 11:04:30,642 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2318f207 >2014-09-17 11:04:30,854 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 2318f207 >2014-09-17 11:04:30,854 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] Host rack storage connection was succeeded >2014-09-17 11:04:30,866 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-1) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 7e574aa >2014-09-17 11:04:31,724 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-1) FINISH, ConnectStoragePoolVDSCommand, log id: 7e574aa >2014-09-17 11:04:31,737 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 65918a1a >2014-09-17 11:04:31,795 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-47) [52ca6744] FINISH, SetMOMPolicyParametersVDSCommand, log id: 65918a1a >2014-09-17 11:04:31,811 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [52ca6744] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-17 11:04:31,822 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [52ca6744] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-17 11:04:31,830 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [52ca6744] Correlation ID: 7baa3cbd, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-17 11:04:31,863 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-47) [268bfe60] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-17 11:04:35,383 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-37) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-17 11:04:35,389 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-37) starting spm on vds rack, storage pool Default, prevId -1, LVER 2 >2014-09-17 11:04:35,391 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-37) START, SpmStartVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=2, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 130bd27c >2014-09-17 11:04:35,399 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-37) spmStart polling started: taskId = 4c66eb1f-9626-4007-92dc-0da812e4327f >2014-09-17 11:04:56,504 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-37) spmStart polling ended: taskId = 4c66eb1f-9626-4007-92dc-0da812e4327f task status = finished >2014-09-17 11:04:56,508 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-37) spmStart polling ended, spm status: SPM >2014-09-17 11:04:56,510 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-37) START, HSMClearTaskVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, taskId=4c66eb1f-9626-4007-92dc-0da812e4327f), log id: 725f7e21 >2014-09-17 11:04:56,515 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, HSMClearTaskVDSCommand, log id: 725f7e21 >2014-09-17 11:04:56,515 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-37) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@195e4878, log id: 130bd27c >2014-09-17 11:04:56,520 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-37) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-17 11:04:56,531 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-37) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-17 11:04:56,538 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-17 11:04:56,540 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 483fe72b >2014-09-17 11:04:56,636 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-17 11:04:56,638 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 29ee433d >2014-09-17 11:04:56,642 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 29ee433d >2014-09-17 11:04:56,642 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-47) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 483fe72b >2014-09-17 11:04:56,643 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-47) Discovered no tasks on Storage Pool Default >2014-09-17 11:54:41,232 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-58) Principal admin::internal synchronized >2014-09-17 11:54:58,682 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-09-17 12:54:41,243 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-17 12:54:58,706 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-09-17 13:03:18,486 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-17 13:03:18,486 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-4) Invalid locale found in configuration: >2014-09-17 13:03:20,543 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-17 13:03:20,544 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-17 13:03:26,956 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginAdminUserCommand internal: false. >2014-09-17 13:03:26,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-17 13:03:43,409 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-5) [75416997] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-17 13:03:43,444 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-17 13:03:43,447 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] START, SetVdsStatusVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 34177f98 >2014-09-17 13:03:43,447 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] VDS rack is spm and moved from up calling resetIrs. >2014-09-17 13:03:43,447 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, ignoreStopFailed = false), log id: 643262ee >2014-09-17 13:03:43,450 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] START, SpmStopVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 5921f0a5 >2014-09-17 13:03:43,455 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-17 13:03:44,684 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] FINISH, SpmStopVDSCommand, log id: 5921f0a5 >2014-09-17 13:03:44,700 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] FINISH, ResetIrsVDSCommand, log id: 643262ee >2014-09-17 13:03:44,719 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] FINISH, SetVdsStatusVDSCommand, log id: 34177f98 >2014-09-17 13:03:44,724 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-17 13:03:44,765 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-85) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = c1aae4b4-6d89-47af-930c-d8b9c6e8323b : rack >2014-09-17 13:03:44,791 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-31) [75416997] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDS >2014-09-17 13:03:44,805 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-31) [75416997] Correlation ID: 75416997, Job ID: 7c8ac546-fc2f-409a-82d6-c7639ebc6255, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-17 13:03:44,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-30) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-17 13:03:44,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-30) Removing vds [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from the domain in maintenance cache >2014-09-17 13:03:44,831 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-30) Removing host(s) [c1aae4b4-6d89-47af-930c-d8b9c6e8323b] from hosts unseen domain report cache >2014-09-17 13:03:44,836 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-85) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 1bad2251 >2014-09-17 13:03:47,281 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-85) FINISH, DisconnectStoragePoolVDSCommand, log id: 1bad2251 >2014-09-17 13:03:47,283 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-85) [3d44c60c] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 13:03:47,291 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-85) [3d44c60c] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2126e0c8 >2014-09-17 13:03:47,550 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-85) [3d44c60c] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 2126e0c8 >2014-09-17 13:04:19,398 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-10) [79c0064a] Lock Acquired to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-17 13:04:19,410 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-22) [79c0064a] Running command: RemoveVdsCommand internal: false. Entities affected : ID: c1aae4b4-6d89-47af-930c-d8b9c6e8323b Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-17 13:04:19,437 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-22) [79c0064a] START, RemoveVdsVDSCommand( HostId = c1aae4b4-6d89-47af-930c-d8b9c6e8323b), log id: 44b86768 >2014-09-17 13:04:19,437 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-22) [79c0064a] vdsManager::disposing >2014-09-17 13:04:19,437 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-22) [79c0064a] FINISH, RemoveVdsVDSCommand, log id: 44b86768 >2014-09-17 13:04:19,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-22) [79c0064a] Correlation ID: 79c0064a, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-17 13:04:19,445 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-22) [79c0064a] Lock freed to object EngineLock [exclusiveLocks= key: c1aae4b4-6d89-47af-930c-d8b9c6e8323b value: VDS >, sharedLocks= ] >2014-09-17 13:06:44,388 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-22) [191d6f35] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues', log id: 1bea6d65: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues' > at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:183) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:32) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.executeVDSCommand(IrsBrokerCommand.java:156) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:56) [vdsbroker.jar:] > at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] > at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:413) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData.proceedStoragePoolStats(IrsProxyData.java:293) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData._updatingTimer_Elapsed(IrsProxyData.java:187) [vdsbroker.jar:] > at sun.reflect.GeneratedMethodAccessor74.invoke(Unknown Source) [:1.7.0_65] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_65] > at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_65] > at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:] > at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:] > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:] > >2014-09-17 13:06:54,401 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-16) [41ad0916] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-17 13:06:54,405 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-16) [41ad0916] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-17 13:06:54,411 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-16) [6af1de4] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-17 13:06:54,427 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-16) [6af1de4] Correlation ID: 6af1de4, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-17 13:54:41,254 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-88) Principal admin::internal synchronized >2014-09-17 13:54:58,723 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-17 14:54:41,263 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-17 14:54:58,726 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) [384f2971] Attempting to update VMs/Templates Ovf. >2014-09-17 15:54:41,273 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-18) Principal admin::internal synchronized >2014-09-17 15:54:58,729 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-22) [191d6f35] Attempting to update VMs/Templates Ovf. >2014-09-17 16:54:41,283 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-60) Principal admin::internal synchronized >2014-09-17 16:54:58,731 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-17 17:54:41,292 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-09-17 17:54:58,734 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) [6af1de4] Attempting to update VMs/Templates Ovf. >2014-09-17 18:54:41,301 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-37) Principal admin::internal synchronized >2014-09-17 18:54:58,737 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-36) Attempting to update VMs/Templates Ovf. >2014-09-17 19:54:41,310 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-11) Principal admin::internal synchronized >2014-09-17 19:54:58,739 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-09-17 20:54:41,320 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-15) Principal admin::internal synchronized >2014-09-17 20:54:58,742 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-17 21:54:41,330 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-17 21:54:58,745 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-56) Attempting to update VMs/Templates Ovf. >2014-09-17 22:54:41,340 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-3) Principal admin::internal synchronized >2014-09-17 22:54:58,748 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) Attempting to update VMs/Templates Ovf. >2014-09-17 23:54:41,351 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-39) Principal admin::internal synchronized >2014-09-17 23:54:58,751 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-09-18 00:54:41,361 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-09-18 00:54:58,754 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) [6af1de4] Attempting to update VMs/Templates Ovf. >2014-09-18 01:54:41,370 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-5) Principal admin::internal synchronized >2014-09-18 01:54:58,757 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-09-18 02:54:41,380 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-76) Principal admin::internal synchronized >2014-09-18 02:54:58,759 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-59) [384f2971] Attempting to update VMs/Templates Ovf. >2014-09-18 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-68) Start deleteAgedOutAuditLogs >2014-09-18 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-93) Start deleteAgedOutCommandEntities >2014-09-18 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-68) Finished deleteAgedOutAuditLogs >2014-09-18 03:35:35,008 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-93) Finished deleteAgedOutCommandEntities >2014-09-18 03:54:41,389 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-95) Principal admin::internal synchronized >2014-09-18 03:54:58,762 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-18 04:54:41,398 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-18 04:54:58,764 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-09-18 05:54:41,410 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-18) Principal admin::internal synchronized >2014-09-18 05:54:58,766 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-09-18 06:54:41,419 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-69) [2589e2b6] Principal admin::internal synchronized >2014-09-18 06:54:58,769 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-18 07:54:41,427 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-53) Principal admin::internal synchronized >2014-09-18 07:54:58,771 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update VMs/Templates Ovf. >2014-09-18 08:54:41,436 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-63) Principal admin::internal synchronized >2014-09-18 08:54:58,774 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) [268bfe60] Attempting to update VMs/Templates Ovf. >2014-09-18 09:15:18,115 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-2) [bceb149] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,115 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-7) [49b74727] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,121 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-6) [2d5675aa] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,123 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-9) [7470327] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,125 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-3) [ab98ed] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,126 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-4) [7931e6e3] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [49b74727] Correlation ID: 49b74727, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,167 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) [bceb149] Correlation ID: bceb149, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [7931e6e3] Correlation ID: 7931e6e3, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [2d5675aa] Correlation ID: 2d5675aa, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,183 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [7470327] Correlation ID: 7470327, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,187 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-3) [ab98ed] Correlation ID: ab98ed, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,197 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-8) [41748a29] Running command: LogoutUserCommand internal: false. >2014-09-18 09:15:18,208 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [41748a29] Correlation ID: 41748a29, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-18 09:15:18,242 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-18 09:15:18,242 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-1) Invalid locale found in configuration: >2014-09-18 09:42:08,502 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-18 09:42:08,502 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-18 09:54:41,447 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-50) Principal admin::internal synchronized >2014-09-18 09:54:58,776 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2014-09-18 09:55:37,301 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginAdminUserCommand internal: false. >2014-09-18 09:55:37,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-18 09:55:37,386 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-4) Running command: LoginUserCommand internal: false. >2014-09-18 10:01:02,949 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-5) [712b682f] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-18 10:01:03,014 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-18 10:01:03,027 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-18 10:01:03,029 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] START, RemoveVdsVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 1d4ef118 >2014-09-18 10:01:03,029 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] FINISH, RemoveVdsVDSCommand, log id: 1d4ef118 >2014-09-18 10:01:03,031 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] START, AddVdsVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 75a9e798 >2014-09-18 10:01:03,031 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] AddVds - entered , starting logic to add VDS 4d573f59-d436-41c0-b7ba-9ef8cf990919 >2014-09-18 10:01:03,033 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] AddVds - VDS 4d573f59-d436-41c0-b7ba-9ef8cf990919 was added, will try to add it to the resource manager >2014-09-18 10:01:03,033 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-5) [13ec0a04] Entered VdsManager constructor >2014-09-18 10:01:03,046 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-5) [13ec0a04] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-18 10:01:03,047 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-5) [13ec0a04] VDS 4d573f59-d436-41c0-b7ba-9ef8cf990919 was added to the Resource Manager >2014-09-18 10:01:03,047 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-5) [13ec0a04] FINISH, AddVdsVDSCommand, log id: 75a9e798 >2014-09-18 10:01:03,060 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [13ec0a04] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-18 10:01:03,088 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [13ec0a04] Correlation ID: 712b682f, Job ID: 8e5797f2-0863-4b05-a047-00bf994cbad2, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-18 10:01:03,095 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] Lock Acquired to object EngineLock [exclusiveLocks= key: 4d573f59-d436-41c0-b7ba-9ef8cf990919 value: VDS >, sharedLocks= ] >2014-09-18 10:01:03,109 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-18 10:01:03,110 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] Before Installation host 4d573f59-d436-41c0-b7ba-9ef8cf990919, rack >2014-09-18 10:01:03,115 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 1a6964be >2014-09-18 10:01:03,123 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] FINISH, SetVdsStatusVDSCommand, log id: 1a6964be >2014-09-18 10:01:03,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-1) [41e33100] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e >2014-09-18 10:01:03,179 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-1) [41e33100] Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: ae:fa:9d:be:6f:b2:a2:ef:f2:84:03:be:41:19:9f:1e. >2014-09-18 10:01:03,247 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-1) [41e33100] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-18 10:01:03,264 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-1) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-18 10:01:04,322 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-18 10:01:04,331 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-18 10:01:04,343 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-18 10:01:04,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-18 10:01:04,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-18 10:01:04,382 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-18 10:01:06,059 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-77) Initializing Host: rack >2014-09-18 10:01:11,429 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-18 10:01:11,440 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-18 10:01:11,440 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-18 10:01:11,452 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-18 10:01:11,810 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-18 10:01:11,823 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-18 10:01:11,836 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140918100103-fe5sge.log' >2014-09-18 10:01:11,845 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140918100103-fe5sge.log'. >2014-09-18 10:01:11,934 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-18 10:01:11,964 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-18 10:01:12,071 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-18 10:01:12,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-18 10:01:12,103 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-18 10:01:12,114 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-18 10:01:12,143 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-18 10:01:12,155 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-18 10:01:12,369 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-18 10:01:12,380 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-18 10:01:12,415 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-18 10:01:12,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-18 10:01:12,426 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-18 10:01:12,434 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-18 10:01:18,584 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-18 10:01:18,594 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-18 10:01:18,644 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-18 10:01:18,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Setting time. >2014-09-18 10:01:20,301 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-18 10:01:20,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-18 10:01:21,593 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-18 10:01:21,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-18 10:01:21,661 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-18 10:01:21,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-18 10:01:22,581 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-18 10:01:22,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-18 10:01:22,854 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-18 10:01:22,863 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-18 10:01:25,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-18 10:01:25,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-18 10:01:25,032 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140918100125-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-41e33100.log' >2014-09-18 10:01:25,043 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140918100125-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-41e33100.log'. >2014-09-18 10:01:25,493 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-18 10:01:25,505 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 41e33100, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-18 10:01:25,633 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-18 10:01:25,733 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-1) [41e33100] Engine managed to communicate with VDSM agent on host rack >2014-09-18 10:01:26,209 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-1) [41e33100] The management network rhevm is already configured on host rack >2014-09-18 10:01:26,212 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 7de8b4df >2014-09-18 10:01:26,228 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] FINISH, SetVdsStatusVDSCommand, log id: 7de8b4df >2014-09-18 10:01:26,228 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] After Installation host rack, VDS >2014-09-18 10:01:26,243 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-1) [41e33100] Correlation ID: 41e33100, Job ID: 8e5797f2-0863-4b05-a047-00bf994cbad2, Call Stack: null, Custom Event ID: -1, Message: Host rack installed >2014-09-18 10:01:26,253 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-1) [41e33100] Lock freed to object EngineLock [exclusiveLocks= key: 4d573f59-d436-41c0-b7ba-9ef8cf990919 value: VDS >, sharedLocks= ] >2014-09-18 10:01:27,138 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) [2a52126a] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vds=Host[rack,4d573f59-d436-41c0-b7ba-9ef8cf990919]), log id: d6feb80 >2014-09-18 10:01:27,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) [2a52126a] FINISH, GetHardwareInfoVDSCommand, log id: d6feb80 >2014-09-18 10:01:27,213 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-34) [5d2716a0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-18 10:01:27,252 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) [5d2716a0] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vds=Host[rack,4d573f59-d436-41c0-b7ba-9ef8cf990919]), log id: 60f0d225 >2014-09-18 10:01:27,260 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-34) [5d2716a0] FINISH, GetHardwareInfoVDSCommand, log id: 60f0d225 >2014-09-18 10:01:27,316 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-34) [782600bc] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-18 10:01:27,329 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-18 10:01:27,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 289723d9 >2014-09-18 10:01:27,481 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 289723d9 >2014-09-18 10:01:27,481 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] Host rack storage connection was succeeded >2014-09-18 10:01:27,499 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-33) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vdsId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 717e7a85 >2014-09-18 10:01:27,995 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-33) FINISH, ConnectStoragePoolVDSCommand, log id: 717e7a85 >2014-09-18 10:01:28,005 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 22aa1bc7 >2014-09-18 10:01:28,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-34) [60f87471] FINISH, SetMOMPolicyParametersVDSCommand, log id: 22aa1bc7 >2014-09-18 10:01:28,075 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [60f87471] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-18 10:01:28,087 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [60f87471] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-18 10:01:28,095 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-34) [60f87471] Correlation ID: 782600bc, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-18 10:01:28,122 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-34) [4bfa72fc] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-18 10:01:28,129 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-34) [5894b823] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-18 10:01:36,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-49) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-18 10:01:36,140 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-49) starting spm on vds rack, storage pool Default, prevId -1, LVER 3 >2014-09-18 10:01:36,141 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-49) START, SpmStartVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=3, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 5eea18e0 >2014-09-18 10:01:36,150 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-49) spmStart polling started: taskId = ccd6f482-6cb9-4410-b3ca-a4a58ee18720 >2014-09-18 10:01:57,256 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-49) spmStart polling ended: taskId = ccd6f482-6cb9-4410-b3ca-a4a58ee18720 task status = finished >2014-09-18 10:01:57,260 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-49) spmStart polling ended, spm status: SPM >2014-09-18 10:01:57,263 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-49) START, HSMClearTaskVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, taskId=ccd6f482-6cb9-4410-b3ca-a4a58ee18720), log id: 16e77f16 >2014-09-18 10:01:57,266 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, HSMClearTaskVDSCommand, log id: 16e77f16 >2014-09-18 10:01:57,266 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-49) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@7cff2cb8, log id: 5eea18e0 >2014-09-18 10:01:57,272 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-49) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-18 10:01:57,282 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-49) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-18 10:01:57,287 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-18 10:01:57,292 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 36eee719 >2014-09-18 10:01:57,398 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-18 10:01:57,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 70fe4fdd >2014-09-18 10:01:57,405 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 70fe4fdd >2014-09-18 10:01:57,405 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 36eee719 >2014-09-18 10:01:57,405 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-3) Discovered no tasks on Storage Pool Default >2014-09-18 10:54:41,456 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-75) Principal admin::internal synchronized >2014-09-18 10:54:58,778 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-09-18 11:54:41,467 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-62) Principal admin::internal synchronized >2014-09-18 11:54:58,798 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-09-18 12:54:41,478 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-67) Principal admin::internal synchronized >2014-09-18 12:54:58,820 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-18 13:37:11,336 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-6) [3070602b] Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-18 13:37:11,421 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [3070602b] START, IsVmDuringInitiatingVDSCommand( vmId = 349f8284-643b-4d9e-b208-aff7af2b5fb3), log id: 27abe033 >2014-09-18 13:37:11,421 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-6) [3070602b] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 27abe033 >2014-09-18 13:37:11,509 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] Running command: RunVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-18 13:37:11,620 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] START, UpdateVmDynamicDataVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@b28da530), log id: 265d8051 >2014-09-18 13:37:11,640 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] FINISH, UpdateVmDynamicDataVDSCommand, log id: 265d8051 >2014-09-18 13:37:11,652 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] START, CreateVmVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 7152d61e >2014-09-18 13:37:11,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] START, CreateVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 38dfa7bd >2014-09-18 13:37:11,741 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, type=pci, slot=0x02, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=798508b7-1373-4465-a7b4-1c5f25d38bd6}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=fa22ef87-6e17-4c22-b6fe-50d5d4ae9758}, {shared=false, index=0, volumeID=455469db-f6b6-4e6a-b906-8dc04ce3907a, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=72c1d2e5-3906-472a-884a-beaf8178a948, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=72c1d2e5-3906-472a-884a-beaf8178a948}, {nicModel=pv, address={bus=0x00, domain=0x0000, type=pci, slot=0x03, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:90, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92a2107d-cec5-47a5-96bf-3dd4ec81fb6c}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3838190b-abba-4a39-b5b4-3845d56b684b}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=1a591966-8fde-4c36-a832-8c75d400b1c1}],vmName=f20,cpuType=SandyBridge,fileTransferEnable=true >2014-09-18 13:37:11,800 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] FINISH, CreateVDSCommand, log id: 38dfa7bd >2014-09-18 13:37:11,815 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 7152d61e >2014-09-18 13:37:11,815 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-34) [3070602b] Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-18 13:37:11,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-34) [3070602b] Correlation ID: 3070602b, Job ID: 1e9455fa-7602-4602-944d-dd64394e1a6e, Call Stack: null, Custom Event ID: -1, Message: VM f20 was started by admin (Host: rack). >2014-09-18 13:37:14,397 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-26) START, DestroyVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=0, gracefully=false, reason=), log id: 5e36f1cf >2014-09-18 13:37:14,410 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, DestroyVDSCommand, log id: 5e36f1cf >2014-09-18 13:37:14,430 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-26) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM f20 is down with error. Exit message: Child quit during startup handshake: Input/output error. >2014-09-18 13:37:14,430 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Running on vds during rerun failed vm: null >2014-09-18 13:37:14,433 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) VM f20 (349f8284-643b-4d9e-b208-aff7af2b5fb3) is running in db and not running in VDS rack >2014-09-18 13:37:14,433 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-26) START, FullListVdsCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vds=Host[rack,4d573f59-d436-41c0-b7ba-9ef8cf990919], vmIds=[349f8284-643b-4d9e-b208-aff7af2b5fb3]), log id: 60b73426 >2014-09-18 13:37:14,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-26) FINISH, FullListVdsCommand, return: [], log id: 60b73426 >2014-09-18 13:37:14,446 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-26) Rerun vm 349f8284-643b-4d9e-b208-aff7af2b5fb3. Called from vds rack >2014-09-18 13:37:14,466 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-43) Correlation ID: 3070602b, Job ID: 1e9455fa-7602-4602-944d-dd64394e1a6e, Call Stack: null, Custom Event ID: -1, Message: Failed to run VM f20 on Host rack. >2014-09-18 13:37:14,480 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-43) Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-18 13:37:14,567 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, IsVmDuringInitiatingVDSCommand( vmId = 349f8284-643b-4d9e-b208-aff7af2b5fb3), log id: 164ed6a3 >2014-09-18 13:37:14,568 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 164ed6a3 >2014-09-18 13:37:14,580 WARN [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-43) CanDoAction of action RunVm failed. Reasons:VAR__ACTION__RUN,VAR__TYPE__VM,VAR__ACTION__RUN,VAR__TYPE__VM,VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_NO_HOSTS >2014-09-18 13:37:14,581 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-43) Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-18 13:37:14,605 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-43) Correlation ID: 3070602b, Job ID: 1e9455fa-7602-4602-944d-dd64394e1a6e, Call Stack: null, Custom Event ID: -1, Message: Failed to run VM f20 (User: admin). >2014-09-18 13:37:14,614 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-1) [72aae3d4] Running command: ProcessDownVmCommand internal: true. >2014-09-18 13:37:41,425 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-18 13:37:41,425 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-18 13:54:41,488 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-18 13:54:58,841 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-18 14:54:41,498 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-33) Principal admin::internal synchronized >2014-09-18 14:54:58,862 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-09-18 15:54:41,509 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-18 15:54:58,884 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2014-09-18 16:54:41,518 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-18 16:54:58,904 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) Attempting to update VMs/Templates Ovf. >2014-09-18 17:54:41,528 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-18 17:54:58,924 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-09-18 18:54:41,540 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-64) [435a3a3] Principal admin::internal synchronized >2014-09-18 18:54:58,944 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update VMs/Templates Ovf. >2014-09-18 19:54:41,551 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-66) Principal admin::internal synchronized >2014-09-18 19:54:58,962 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-18 20:54:41,559 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-72) Principal admin::internal synchronized >2014-09-18 20:54:58,985 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-18 21:54:41,569 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-42) Principal admin::internal synchronized >2014-09-18 21:54:59,006 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-18 22:54:41,579 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-19) Principal admin::internal synchronized >2014-09-18 22:54:59,029 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-18 23:54:41,588 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-18 23:54:59,049 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-09-19 00:54:41,596 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-19 00:54:59,069 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) Attempting to update VMs/Templates Ovf. >2014-09-19 01:54:41,605 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-19 01:54:59,089 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-70) Attempting to update VMs/Templates Ovf. >2014-09-19 02:54:41,614 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-16) [6af1de4] Principal admin::internal synchronized >2014-09-19 02:54:59,108 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) [3fa8ffdd] Attempting to update VMs/Templates Ovf. >2014-09-19 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-72) Start deleteAgedOutAuditLogs >2014-09-19 03:35:35,003 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-71) Start deleteAgedOutCommandEntities >2014-09-19 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-72) Finished deleteAgedOutAuditLogs >2014-09-19 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-71) Finished deleteAgedOutCommandEntities >2014-09-19 03:54:41,625 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) [3fa8ffdd] Principal admin::internal synchronized >2014-09-19 03:54:59,127 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-09-19 04:54:41,637 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-3) Principal admin::internal synchronized >2014-09-19 04:54:59,151 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-09-19 05:54:41,648 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-98) Principal admin::internal synchronized >2014-09-19 05:54:59,175 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-19 06:54:41,659 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-88) Principal admin::internal synchronized >2014-09-19 06:54:59,198 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-66) Attempting to update VMs/Templates Ovf. >2014-09-19 07:54:41,670 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) [5894b823] Principal admin::internal synchronized >2014-09-19 07:54:59,221 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) [3d44c60c] Attempting to update VMs/Templates Ovf. >2014-09-19 08:54:41,682 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-14) Principal admin::internal synchronized >2014-09-19 08:54:59,244 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-12) [a30cd7c] Attempting to update VMs/Templates Ovf. >2014-09-19 09:54:41,692 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-48) Principal admin::internal synchronized >2014-09-19 09:54:59,264 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-19 10:54:41,704 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-69) [2589e2b6] Principal admin::internal synchronized >2014-09-19 10:54:59,287 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-19 11:03:25,609 ERROR [org.ovirt.engine.core.aaa.filters.BasicAuthenticationFilter] (ajp-/127.0.0.1:8702-6) Error in obtaining profile null >2014-09-19 11:03:43,990 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-2) Running command: LoginUserCommand internal: false. >2014-09-19 11:03:44,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-2) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-19 11:03:49,629 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2014-09-19 11:03:49,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-19 11:03:49,751 INFO [org.ovirt.engine.core.bll.aaa.LogoutBySessionCommand] (ajp-/127.0.0.1:8702-5) [25bb080f] Running command: LogoutBySessionCommand internal: false. >2014-09-19 11:03:49,755 INFO [org.ovirt.engine.core.bll.aaa.LogoutUserCommand] (ajp-/127.0.0.1:8702-5) [18588672] Running command: LogoutUserCommand internal: false. >2014-09-19 11:03:49,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) [18588672] Correlation ID: 18588672, Call Stack: null, Custom Event ID: -1, Message: User admin logged out. >2014-09-19 11:03:49,780 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: 25bb080f, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-19 11:05:15,363 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-19 11:05:15,375 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-19 11:05:15,376 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-19 11:05:15,376 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-19 11:05:15,376 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-19 11:05:15,376 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-19 11:05:15,377 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-19 11:05:15,377 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-19 11:05:15,377 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-19 11:05:15,378 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-19 11:05:15,378 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-19 11:05:15,388 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-19 11:05:15,389 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-19 11:05:15,390 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-19 11:05:15,412 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-19 11:05:15,413 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-19 11:05:15,414 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-19 11:05:15,415 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-19 11:05:15,417 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-09-19 11:05:15,417 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-09-19 11:05:15,417 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-09-19 11:05:15,417 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-19 11:05:15,418 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-19 11:05:15,419 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-19 11:05:15,419 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-19 11:05:15,419 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-19 11:05:15,419 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-19 11:05:15,419 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-19 11:05:24,497 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Start initializing Backend >2014-09-19 11:05:25,115 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 52) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-19 11:05:25,303 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Running ovirt-engine 3.5.0-0.11.beta.el6ev >2014-09-19 11:05:25,304 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Start initializing dictionaries >2014-09-19 11:05:25,308 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 52) Finished initializing dictionaries >2014-09-19 11:05:25,312 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing AuditLogCleanupManager >2014-09-19 11:05:25,312 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-19 11:05:25,355 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing AuditLogCleanupManager >2014-09-19 11:05:25,356 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Start initializing CommandEntityCleanupManager >2014-09-19 11:05:25,356 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-19 11:05:25,358 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 52) Finished initializing CommandEntityCleanupManager >2014-09-19 11:05:25,360 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Start initializing TagsDirector >2014-09-19 11:05:25,371 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Tag root added to tree >2014-09-19 11:05:25,398 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 52) Finished initializing TagsDirector >2014-09-19 11:05:25,399 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Start initializing IsoDomainListSyncronizer >2014-09-19 11:05:25,412 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 52) Finished initializing IsoDomainListSyncronizer >2014-09-19 11:05:25,495 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Completed initializing handlers >2014-09-19 11:05:25,533 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl >2014-09-19 11:05:25,551 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-19 11:05:25,552 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl >2014-09-19 11:05:25,552 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Start initializing ErrorTranslatorImpl >2014-09-19 11:05:25,553 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 52) Finished initializing ErrorTranslatorImpl >2014-09-19 11:05:25,553 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 52) Mark incomplete jobs as UNKNOWN >2014-09-19 11:05:25,610 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Start initializing JobRepositoryCleanupManager >2014-09-19 11:05:25,618 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 52) Finished initializing JobRepositoryCleanupManager >2014-09-19 11:05:25,618 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Start initializing AutoRecoveryManager >2014-09-19 11:05:25,619 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 52) Finished initializing AutoRecoveryManager >2014-09-19 11:05:25,622 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Start initializing ExecutionMessageDirector >2014-09-19 11:05:25,629 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 52) Finished initializing ExecutionMessageDirector >2014-09-19 11:05:25,810 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'builtin-authn-internal' >2014-09-19 11:05:25,817 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' loaded >2014-09-19 11:05:25,820 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Loading extension 'internal' >2014-09-19 11:05:25,823 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' loaded >2014-09-19 11:05:25,824 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'builtin-authn-internal' >2014-09-19 11:05:25,824 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'builtin-authn-internal' initialized >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Initializing extension 'internal' >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Extension 'internal' initialized >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Start of enabled extensions list >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-19 11:05:25,825 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 52) End of enabled extensions list >2014-09-19 11:05:26,053 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 48) Start initializing DbUserCacheManager >2014-09-19 11:05:26,058 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 48) Finished initializing DbUserCacheManager >2014-09-19 11:05:26,109 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 48) Initialization of AsyncTaskManager completed successfully. >2014-09-19 11:05:26,112 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Start initializing ResourceManager >2014-09-19 11:05:26,204 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-19 11:05:26,402 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 48) Entered VdsManager constructor >2014-09-19 11:05:26,483 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 48) Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-19 11:05:26,783 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) VDS 4d573f59-d436-41c0-b7ba-9ef8cf990919 was added to the Resource Manager >2014-09-19 11:05:26,826 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 48) Finished initializing ResourceManager >2014-09-19 11:05:26,835 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 48) Initialization of OvfDataUpdater completed successfully. >2014-09-19 11:05:26,836 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start scheduling to enable vds load balancer >2014-09-19 11:05:26,837 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished scheduling to enable vds load balancer >2014-09-19 11:05:26,837 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Start HA Reservation check >2014-09-19 11:05:26,837 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Finished HA Reservation check >2014-09-19 11:05:26,847 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Start initializing MacPoolManagerRanges >2014-09-19 11:05:26,864 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init VM custom properties utilities >2014-09-19 11:05:26,865 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 48) Init device custom properties utilities >2014-09-19 11:05:26,876 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initializing Scheduling manager >2014-09-19 11:05:26,889 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-6-thread-1) Finished initializing. Available MACs in pool: 63 >2014-09-19 11:05:26,936 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) External scheduler disabled, discovery skipped >2014-09-19 11:05:26,936 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 48) Initialized Scheduling manager >2014-09-19 11:05:26,936 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) Initializing DWH Heart Beat >2014-09-19 11:05:26,938 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 48) DWH Heart Beat initialized >2014-09-19 11:05:30,018 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 11:05:30,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, vds=Host[rack,4d573f59-d436-41c0-b7ba-9ef8cf990919]), log id: 856cbdf >2014-09-19 11:05:30,833 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-7) FINISH, GetHardwareInfoVDSCommand, log id: 856cbdf >2014-09-19 11:05:31,374 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-7) Initializing Host: rack >2014-09-19 11:05:31,622 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-7) [31de4b0a] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-19 11:05:37,003 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-21) [adb8dca] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-19 11:05:37,017 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [adb8dca] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-19 11:05:37,030 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-21) [adb8dca] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : null >2014-09-19 11:05:37,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-19 11:05:37,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-19 11:05:37,077 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-19 11:05:37,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-19 11:05:37,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-19 11:05:37,078 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,079 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-19 11:05:37,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,082 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,083 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-19 11:05:37,542 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] Correlation ID: adb8dca, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-19 11:05:37,739 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [adb8dca] hostFromVds::selectedVds - rack, spmStatus SPM, storage pool Default >2014-09-19 11:05:37,755 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-21) [adb8dca] Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-19 11:05:37,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-21) [adb8dca] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-19 11:05:37,779 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 11:05:37,997 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 599975f0 >2014-09-19 11:05:38,021 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-19 11:05:38,061 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 3f163cd6 >2014-09-19 11:05:38,069 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 3f163cd6 >2014-09-19 11:05:38,069 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 599975f0 >2014-09-19 11:05:38,069 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-6) [adb8dca] Discovered no tasks on Storage Pool Default >2014-09-19 12:05:26,257 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-19 12:05:26,837 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-09-19 13:05:26,271 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-3) Principal admin::internal synchronized >2014-09-19 13:05:26,946 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-19 14:05:26,293 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-53) Principal admin::internal synchronized >2014-09-19 14:05:26,985 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-67) Attempting to update VMs/Templates Ovf. >2014-09-19 14:34:27,046 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-2) Invalid locale found in configuration: >2014-09-19 14:34:27,046 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-2) Invalid locale found in configuration: >2014-09-19 14:34:33,518 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-19 14:34:33,523 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-19 14:34:33,539 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-19 14:34:33,540 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-19 14:34:41,424 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginAdminUserCommand internal: false. >2014-09-19 14:34:41,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-19 14:34:45,256 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginUserCommand internal: false. >2014-09-19 14:34:57,916 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-10) [426451d3] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-19 14:34:58,040 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-19 14:34:58,046 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] START, SetVdsStatusVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 630765e6 >2014-09-19 14:34:58,047 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] VDS rack is spm and moved from up calling resetIrs. >2014-09-19 14:34:58,048 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, ignoreStopFailed = false), log id: 782eabc0 >2014-09-19 14:34:58,053 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] START, SpmStopVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 17ecb6e3 >2014-09-19 14:34:58,062 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-19 14:34:59,175 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] FINISH, SpmStopVDSCommand, log id: 17ecb6e3 >2014-09-19 14:34:59,189 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] FINISH, ResetIrsVDSCommand, log id: 782eabc0 >2014-09-19 14:34:59,211 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] FINISH, SetVdsStatusVDSCommand, log id: 630765e6 >2014-09-19 14:34:59,309 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-19 14:34:59,436 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-24) [426451d3] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDS >2014-09-19 14:34:59,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-24) [426451d3] Correlation ID: 426451d3, Job ID: 0a0e4b81-21e1-4c9f-8bf5-f155569f2942, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-19 14:35:00,850 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-30) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 4d573f59-d436-41c0-b7ba-9ef8cf990919 : rack >2014-09-19 14:35:00,893 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-26) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-19 14:35:00,894 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-26) Removing vds [4d573f59-d436-41c0-b7ba-9ef8cf990919] from the domain in maintenance cache >2014-09-19 14:35:00,894 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-26) Removing host(s) [4d573f59-d436-41c0-b7ba-9ef8cf990919] from hosts unseen domain report cache >2014-09-19 14:35:00,900 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-30) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: b3190c6 >2014-09-19 14:35:01,902 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-30) FINISH, DisconnectStoragePoolVDSCommand, log id: b3190c6 >2014-09-19 14:35:01,908 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-19 14:35:01,965 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [3d8a45b] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 747e0de5 >2014-09-19 14:35:01,970 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-29) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-19 14:35:01,977 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-29) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-19 14:35:01,988 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-29) [3775fd20] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-19 14:35:02,053 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-29) [3775fd20] Correlation ID: 3775fd20, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-19 14:35:02,306 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-30) [3d8a45b] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 747e0de5 >2014-09-19 14:35:05,319 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-6) [275cb7b8] Lock Acquired to object EngineLock [exclusiveLocks= key: 4d573f59-d436-41c0-b7ba-9ef8cf990919 value: VDS >, sharedLocks= ] >2014-09-19 14:35:05,340 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] Running command: RemoveVdsCommand internal: false. Entities affected : ID: 4d573f59-d436-41c0-b7ba-9ef8cf990919 Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-19 14:35:05,430 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] START, RemoveVdsVDSCommand( HostId = 4d573f59-d436-41c0-b7ba-9ef8cf990919), log id: 1d908bf4 >2014-09-19 14:35:05,431 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] vdsManager::disposing >2014-09-19 14:35:05,440 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] FINISH, RemoveVdsVDSCommand, log id: 1d908bf4 >2014-09-19 14:35:05,462 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] Correlation ID: 275cb7b8, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-19 14:35:05,462 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-28) [275cb7b8] Lock freed to object EngineLock [exclusiveLocks= key: 4d573f59-d436-41c0-b7ba-9ef8cf990919 value: VDS >, sharedLocks= ] >2014-09-19 15:05:26,306 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-16) Principal admin::internal synchronized >2014-09-19 15:05:27,028 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-09-19 15:36:28,213 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-8) [54e3d54a] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-19 15:36:28,353 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-19 15:36:28,382 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-19 15:36:28,386 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] START, RemoveVdsVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 3f4f77b9 >2014-09-19 15:36:28,386 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] FINISH, RemoveVdsVDSCommand, log id: 3f4f77b9 >2014-09-19 15:36:28,389 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] START, AddVdsVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 47cf47de >2014-09-19 15:36:28,389 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] AddVds - entered , starting logic to add VDS e58f9fdb-4a02-44ea-be4d-e2162edb495c >2014-09-19 15:36:28,391 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] AddVds - VDS e58f9fdb-4a02-44ea-be4d-e2162edb495c was added, will try to add it to the resource manager >2014-09-19 15:36:28,391 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-8) [6d3dc998] Entered VdsManager constructor >2014-09-19 15:36:28,426 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-8) [6d3dc998] Initialize vdsBroker (dell-r210ii-16.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-19 15:36:28,428 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-8) [6d3dc998] VDS e58f9fdb-4a02-44ea-be4d-e2162edb495c was added to the Resource Manager >2014-09-19 15:36:28,429 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-8) [6d3dc998] FINISH, AddVdsVDSCommand, log id: 47cf47de >2014-09-19 15:36:28,446 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [6d3dc998] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-19 15:36:28,510 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Lock Acquired to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-19 15:36:28,507 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) [6d3dc998] Correlation ID: 54e3d54a, Job ID: 158dcfa7-92df-40dd-8521-85cc8a164bae, Call Stack: null, Custom Event ID: -1, Message: Host rack was added by admin. >2014-09-19 15:36:28,531 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-19 15:36:28,543 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Before Installation host e58f9fdb-4a02-44ea-be4d-e2162edb495c, rack >2014-09-19 15:36:28,549 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] START, SetVdsStatusVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5397be11 >2014-09-19 15:36:28,560 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] FINISH, SetVdsStatusVDSCommand, log id: 5397be11 >2014-09-19 15:36:28,638 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 97:93:e2:1d:85:b0:85:66:85:45:96:80:f0:0b:38:73 >2014-09-19 15:36:28,651 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Connected to host dell-r210ii-16.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 97:93:e2:1d:85:b0:85:66:85:45:96:80:f0:0b:38:73. >2014-09-19 15:36:28,716 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Installation of dell-r210ii-16.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-19 15:36:28,741 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-29) SSH execute root@dell-r210ii-16.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-19 15:36:29,840 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-19 15:36:29,850 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Initializing. >2014-09-19 15:36:29,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-19 15:36:29,886 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment setup. >2014-09-19 15:36:29,907 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-19 15:36:29,923 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment packages setup. >2014-09-19 15:36:31,432 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-98) Initializing Host: rack >2014-09-19 15:36:36,445 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-19 15:36:36,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Programs detection. >2014-09-19 15:36:36,457 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-19 15:36:36,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Environment customization. >2014-09-19 15:36:36,860 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Kdump supported >2014-09-19 15:36:36,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Kdump supported. >2014-09-19 15:36:36,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140919153629-1dxtea.log' >2014-09-19 15:36:36,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Logs at host located at: '/tmp/ovirt-host-deploy-20140919153629-1dxtea.log'. >2014-09-19 15:36:37,028 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0034-5310-8052-B4C04F4A354A >2014-09-19 15:36:37,059 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0034-5310-8052-B4C04F4A354A to Host dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-19 15:36:37,177 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-19 15:36:37,202 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Disabling Kdump integration. >2014-09-19 15:36:37,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-19 15:36:37,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Setup validation. >2014-09-19 15:36:37,283 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-19 15:36:37,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Hardware supports virtualization. >2014-09-19 15:36:37,504 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-19 15:36:37,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction setup. >2014-09-19 15:36:37,577 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-19 15:36:37,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-19 15:36:37,601 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-19 15:36:37,617 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Package installation. >2014-09-19 15:36:39,756 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-19 15:36:39,766 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Downloading Packages. >2014-09-19 15:36:41,385 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64 >2014-09-19 15:36:41,397 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Download/Verify: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64. >2014-09-19 15:36:41,397 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-19 15:36:41,413 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Check Package Signatures. >2014-09-19 15:36:41,413 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-19 15:36:41,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Test Transaction. >2014-09-19 15:36:41,465 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-19 15:36:41,474 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Status: Running Transaction. >2014-09-19 15:36:41,664 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum install: 1/1: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64 >2014-09-19 15:36:41,678 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum install: 1/1: 2:qemu-kvm-rhev-tools-0.12.1.2-2.446.el6.x86_64. >2014-09-19 15:36:45,644 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/1: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.446.el6 - u >2014-09-19 15:36:45,657 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Yum Verify: 1/1: qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.446.el6 - u. >2014-09-19 15:36:45,838 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-19 15:36:45,846 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Misc configuration. >2014-09-19 15:36:47,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-19 15:36:47,238 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Enrolling certificate. >2014-09-19 15:36:48,528 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-19 15:36:48,540 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Transaction commit. >2014-09-19 15:36:48,670 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-19 15:36:48,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Closing up. >2014-09-19 15:36:49,678 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-19 15:36:49,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stopping libvirtd. >2014-09-19 15:36:49,712 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-19 15:36:49,723 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Starting vdsm. >2014-09-19 15:36:53,160 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-19 15:36:53,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Pre-termination. >2014-09-19 15:36:53,199 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140919153653-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-3d48d974.log' >2014-09-19 15:36:53,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140919153653-dell-r210ii-16.rhev.lab.eng.brq.redhat.com-3d48d974.log'. >2014-09-19 15:36:53,462 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation dell-r210ii-16.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-19 15:36:53,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 3d48d974, Call Stack: null, Custom Event ID: -1, Message: Installing Host rack. Stage: Termination. >2014-09-19 15:36:53,601 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 15:36:53,602 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-19 15:36:53,604 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-19 15:36:53,604 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-6-thread-30) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:918) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:914) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:405) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:394) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-19 15:36:53,608 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-19 15:36:53,623 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Command PollVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-19 15:36:54,124 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 15:36:54,233 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-29) [3d48d974] Engine managed to communicate with VDSM agent on host rack >2014-09-19 15:36:54,837 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-29) [76e58fa7] Running command: SetupNetworksCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-19 15:36:54,844 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-29) [76e58fa7] START, SetupNetworksVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=e6787fbe-e597-487b-aec8-6af0cf961afa, vdsId=e58f9fdb-4a02-44ea-be4d-e2162edb495c, macAddress=d4:ae:52:c7:0d:8b, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.62.208, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=3bb5321b-426e-456e-bf0a-947b87eb9de8, vdsId=e58f9fdb-4a02-44ea-be4d-e2162edb495c, macAddress=d4:ae:52:c7:0d:8c, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 4550275a >2014-09-19 15:36:54,850 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 15:36:54,850 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-29) [76e58fa7] FINISH, SetupNetworksVDSCommand, log id: 4550275a >2014-09-19 15:37:10,505 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 15:37:10,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-29) [76e58fa7] Correlation ID: 76e58fa7, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-19 15:37:10,732 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-19 15:37:10,735 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] START, SetSafeNetworkConfigVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 34589f42 >2014-09-19 15:37:10,750 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] FINISH, SetSafeNetworkConfigVDSCommand, log id: 34589f42 >2014-09-19 15:37:10,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] Correlation ID: 434bb0b6, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host rack >2014-09-19 15:37:10,777 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] START, SetVdsStatusVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 498722b8 >2014-09-19 15:37:10,798 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] FINISH, SetVdsStatusVDSCommand, log id: 498722b8 >2014-09-19 15:37:10,798 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] After Installation host rack, VDS >2014-09-19 15:37:10,813 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] Correlation ID: 3d48d974, Job ID: 158dcfa7-92df-40dd-8521-85cc8a164bae, Call Stack: null, Custom Event ID: -1, Message: Host rack installed >2014-09-19 15:37:10,822 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-29) [434bb0b6] Lock freed to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-19 15:37:13,532 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-98) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vds=Host[rack,e58f9fdb-4a02-44ea-be4d-e2162edb495c]), log id: 3b9317c8 >2014-09-19 15:37:13,593 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-98) FINISH, GetHardwareInfoVDSCommand, log id: 3b9317c8 >2014-09-19 15:37:13,641 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-98) [4d3c5a1c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-19 15:37:13,685 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-98) [4d3c5a1c] START, GetHardwareInfoVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vds=Host[rack,e58f9fdb-4a02-44ea-be4d-e2162edb495c]), log id: 3bcc35ac >2014-09-19 15:37:13,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-98) [4d3c5a1c] FINISH, GetHardwareInfoVDSCommand, log id: 3bcc35ac >2014-09-19 15:37:13,741 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-98) [40a11513] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-19 15:37:13,771 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-19 15:37:13,788 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3940cf21 >2014-09-19 15:37:14,267 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 3940cf21 >2014-09-19 15:37:14,269 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] Host rack storage connection was succeeded >2014-09-19 15:37:14,300 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-36) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vdsId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 4ce814c5 >2014-09-19 15:37:15,161 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-36) FINISH, ConnectStoragePoolVDSCommand, log id: 4ce814c5 >2014-09-19 15:37:15,181 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 50d5e133 >2014-09-19 15:37:15,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-98) [38d69d26] FINISH, SetMOMPolicyParametersVDSCommand, log id: 50d5e133 >2014-09-19 15:37:15,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) [38d69d26] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-19 15:37:15,265 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) [38d69d26] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-19 15:37:15,275 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-98) [38d69d26] Correlation ID: 40a11513, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-19 15:37:15,324 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-98) [1260f66d] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-19 15:37:15,340 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-98) [153916cb] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-19 15:37:19,235 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-12) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-19 15:37:19,252 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-12) starting spm on vds rack, storage pool Default, prevId -1, LVER 4 >2014-09-19 15:37:19,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-12) START, SpmStartVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=4, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 5f42ee63 >2014-09-19 15:37:19,274 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-12) spmStart polling started: taskId = b6d7bbc0-6387-4469-8baa-50a261f09387 >2014-09-19 15:37:40,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-12) spmStart polling ended: taskId = b6d7bbc0-6387-4469-8baa-50a261f09387 task status = finished >2014-09-19 15:37:40,397 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-12) spmStart polling ended, spm status: SPM >2014-09-19 15:37:40,400 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-12) START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=b6d7bbc0-6387-4469-8baa-50a261f09387), log id: 66a79e0f >2014-09-19 15:37:40,405 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, HSMClearTaskVDSCommand, log id: 66a79e0f >2014-09-19 15:37:40,405 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-12) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@c38ff2c, log id: 5f42ee63 >2014-09-19 15:37:40,412 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-12) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-19 15:37:40,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-19 15:37:40,432 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-19 15:37:40,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-1) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 356a755 >2014-09-19 15:37:40,545 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-1) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-19 15:37:40,547 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-1) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 7d3659cb >2014-09-19 15:37:40,552 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-1) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 7d3659cb >2014-09-19 15:37:40,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-1) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 356a755 >2014-09-19 15:37:40,552 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-1) Discovered no tasks on Storage Pool Default >2014-09-19 16:05:26,317 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-63) Principal admin::internal synchronized >2014-09-19 16:05:27,031 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-09-19 17:05:26,326 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-19 17:05:27,072 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-21) [adb8dca] Attempting to update VMs/Templates Ovf. >2014-09-19 18:05:26,338 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-19 18:05:27,087 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) Attempting to update VMs/Templates Ovf. >2014-09-19 19:05:26,349 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-59) Principal admin::internal synchronized >2014-09-19 19:05:27,111 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-19 20:05:26,361 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-19 20:05:27,143 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-16) Attempting to update VMs/Templates Ovf. >2014-09-19 21:05:26,372 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-19 21:05:27,167 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-85) Attempting to update VMs/Templates Ovf. >2014-09-19 22:05:26,384 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-38) Principal admin::internal synchronized >2014-09-19 22:05:27,191 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-09-19 23:05:26,395 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-19 23:05:27,213 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-97) Attempting to update VMs/Templates Ovf. >2014-09-20 00:05:26,406 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-09-20 00:05:27,237 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-20 01:05:26,417 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-09-20 01:05:27,257 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-20 02:05:26,428 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-76) Principal admin::internal synchronized >2014-09-20 02:05:27,277 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) Attempting to update VMs/Templates Ovf. >2014-09-20 03:05:26,438 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-53) Principal admin::internal synchronized >2014-09-20 03:05:27,299 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-20 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-14) Start deleteAgedOutAuditLogs >2014-09-20 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-96) Start deleteAgedOutCommandEntities >2014-09-20 03:35:35,016 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-14) Finished deleteAgedOutAuditLogs >2014-09-20 03:35:35,022 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-96) Finished deleteAgedOutCommandEntities >2014-09-20 04:05:26,448 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-96) Principal admin::internal synchronized >2014-09-20 04:05:27,322 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-09-20 05:05:26,459 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-09-20 05:05:27,343 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-09-20 06:05:26,471 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-36) Principal admin::internal synchronized >2014-09-20 06:05:27,365 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-20 07:05:26,481 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-62) Principal admin::internal synchronized >2014-09-20 07:05:27,390 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-20 08:05:26,496 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-45) Principal admin::internal synchronized >2014-09-20 08:05:27,413 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-32) Attempting to update VMs/Templates Ovf. >2014-09-20 09:05:26,507 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-23) Principal admin::internal synchronized >2014-09-20 09:05:27,438 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-1) Attempting to update VMs/Templates Ovf. >2014-09-20 10:05:26,517 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-76) Principal admin::internal synchronized >2014-09-20 10:05:27,459 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-49) Attempting to update VMs/Templates Ovf. >2014-09-20 11:05:26,526 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Principal admin::internal synchronized >2014-09-20 11:05:27,482 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-09-20 12:05:26,537 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-09-20 12:05:27,505 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-20 13:05:26,547 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-20 13:05:27,525 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-46) Attempting to update VMs/Templates Ovf. >2014-09-20 14:05:26,561 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-20 14:05:27,552 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-09-20 15:05:26,571 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-49) Principal admin::internal synchronized >2014-09-20 15:05:27,573 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-09-20 16:05:26,585 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-20 16:05:27,595 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-20 17:05:26,595 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-33) Principal admin::internal synchronized >2014-09-20 17:05:27,617 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-9) Attempting to update VMs/Templates Ovf. >2014-09-20 18:05:26,607 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-77) Principal admin::internal synchronized >2014-09-20 18:05:27,635 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-91) Attempting to update VMs/Templates Ovf. >2014-09-20 19:05:26,618 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-6) Principal admin::internal synchronized >2014-09-20 19:05:27,659 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) [3775fd20] Attempting to update VMs/Templates Ovf. >2014-09-20 20:05:26,629 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-20 20:05:27,684 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-23) Attempting to update VMs/Templates Ovf. >2014-09-20 21:05:26,641 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-72) Principal admin::internal synchronized >2014-09-20 21:05:27,705 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-47) Attempting to update VMs/Templates Ovf. >2014-09-20 22:05:26,650 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-26) Principal admin::internal synchronized >2014-09-20 22:05:27,726 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-20 23:05:26,661 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-09-20 23:05:27,746 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-09-21 00:05:26,671 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-55) Principal admin::internal synchronized >2014-09-21 00:05:27,768 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-09-21 01:05:26,683 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-100) Principal admin::internal synchronized >2014-09-21 01:05:27,790 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-65) Attempting to update VMs/Templates Ovf. >2014-09-21 02:05:26,694 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-61) Principal admin::internal synchronized >2014-09-21 02:05:27,811 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-73) Attempting to update VMs/Templates Ovf. >2014-09-21 03:05:26,703 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Principal admin::internal synchronized >2014-09-21 03:05:27,833 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-6) Attempting to update VMs/Templates Ovf. >2014-09-21 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-81) Start deleteAgedOutAuditLogs >2014-09-21 03:35:35,003 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-76) Start deleteAgedOutCommandEntities >2014-09-21 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-76) Finished deleteAgedOutCommandEntities >2014-09-21 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-81) Finished deleteAgedOutAuditLogs >2014-09-21 04:05:26,712 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-87) Principal admin::internal synchronized >2014-09-21 04:05:27,854 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-21 05:05:26,721 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-45) Principal admin::internal synchronized >2014-09-21 05:05:27,876 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-09-21 06:05:26,732 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-15) Principal admin::internal synchronized >2014-09-21 06:05:27,898 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-09-21 07:05:26,743 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-77) Principal admin::internal synchronized >2014-09-21 07:05:27,920 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-09-21 08:05:26,754 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-57) Principal admin::internal synchronized >2014-09-21 08:05:27,942 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-42) Attempting to update VMs/Templates Ovf. >2014-09-21 09:05:26,766 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-33) Principal admin::internal synchronized >2014-09-21 09:05:27,963 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-93) Attempting to update VMs/Templates Ovf. >2014-09-21 10:05:26,776 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) Principal admin::internal synchronized >2014-09-21 10:05:27,982 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-88) Attempting to update VMs/Templates Ovf. >2014-09-21 11:05:26,787 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-21 11:05:28,003 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-78) Attempting to update VMs/Templates Ovf. >2014-09-21 12:05:26,797 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-91) Principal admin::internal synchronized >2014-09-21 12:05:28,025 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-21 13:05:26,809 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-18) Principal admin::internal synchronized >2014-09-21 13:05:28,053 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) [3775fd20] Attempting to update VMs/Templates Ovf. >2014-09-21 14:05:26,819 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-2) Principal admin::internal synchronized >2014-09-21 14:05:28,074 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-98) [153916cb] Attempting to update VMs/Templates Ovf. >2014-09-21 15:05:26,828 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-21 15:05:28,093 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-19) Attempting to update VMs/Templates Ovf. >2014-09-21 16:05:26,839 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-21 16:05:28,114 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-21 17:05:26,848 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-21 17:05:28,138 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-21 18:05:26,859 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-66) Principal admin::internal synchronized >2014-09-21 18:05:28,160 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) Attempting to update VMs/Templates Ovf. >2014-09-21 19:05:26,874 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-28) Principal admin::internal synchronized >2014-09-21 19:05:28,181 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Attempting to update VMs/Templates Ovf. >2014-09-21 20:05:26,885 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-73) Principal admin::internal synchronized >2014-09-21 20:05:28,208 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-62) Attempting to update VMs/Templates Ovf. >2014-09-21 21:05:26,903 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-70) Principal admin::internal synchronized >2014-09-21 21:05:28,229 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) Attempting to update VMs/Templates Ovf. >2014-09-21 22:05:26,914 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) [31de4b0a] Principal admin::internal synchronized >2014-09-21 22:05:28,251 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-09-21 23:05:26,928 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-81) Principal admin::internal synchronized >2014-09-21 23:05:28,272 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-09-22 00:05:26,938 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-45) Principal admin::internal synchronized >2014-09-22 00:05:28,293 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Attempting to update VMs/Templates Ovf. >2014-09-22 01:05:26,955 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-71) Principal admin::internal synchronized >2014-09-22 01:05:28,315 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-22 02:05:26,971 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-18) Principal admin::internal synchronized >2014-09-22 02:05:28,336 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-09-22 03:05:26,979 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-31) Principal admin::internal synchronized >2014-09-22 03:05:28,361 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-15) Attempting to update VMs/Templates Ovf. >2014-09-22 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-76) Start deleteAgedOutAuditLogs >2014-09-22 03:35:35,003 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-19) Start deleteAgedOutCommandEntities >2014-09-22 03:35:35,008 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-76) Finished deleteAgedOutAuditLogs >2014-09-22 03:35:35,008 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-19) Finished deleteAgedOutCommandEntities >2014-09-22 04:05:26,990 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-94) Principal admin::internal synchronized >2014-09-22 04:05:28,381 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-09-22 05:05:27,000 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-09-22 05:05:28,404 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-22 06:05:27,011 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-98) [153916cb] Principal admin::internal synchronized >2014-09-22 06:05:28,421 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-41) Attempting to update VMs/Templates Ovf. >2014-09-22 07:05:27,022 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-85) Principal admin::internal synchronized >2014-09-22 07:05:28,446 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-44) Attempting to update VMs/Templates Ovf. >2014-09-22 08:05:27,031 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-58) Principal admin::internal synchronized >2014-09-22 08:05:28,472 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-09-22 09:05:27,041 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) Principal admin::internal synchronized >2014-09-22 09:05:28,497 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-25) Attempting to update VMs/Templates Ovf. >2014-09-22 10:05:27,051 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) Principal admin::internal synchronized >2014-09-22 10:05:28,517 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-22 10:35:17,135 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-22 10:35:17,135 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-8) Invalid locale found in configuration: >2014-09-22 11:05:27,060 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-22 11:05:28,538 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-28) Attempting to update VMs/Templates Ovf. >2014-09-22 12:05:27,071 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-77) Principal admin::internal synchronized >2014-09-22 12:05:28,560 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-22 13:05:27,079 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-47) Principal admin::internal synchronized >2014-09-22 13:05:28,578 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-53) Attempting to update VMs/Templates Ovf. >2014-09-22 14:05:27,090 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-79) Principal admin::internal synchronized >2014-09-22 14:05:28,596 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-09-22 15:05:27,101 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-22 15:05:28,610 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-22 16:05:27,111 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-43) Principal admin::internal synchronized >2014-09-22 16:05:28,629 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-22 17:05:27,122 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) [31de4b0a] Principal admin::internal synchronized >2014-09-22 17:05:28,648 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-35) Attempting to update VMs/Templates Ovf. >2014-09-22 18:05:27,133 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-09-22 18:05:28,673 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-81) Attempting to update VMs/Templates Ovf. >2014-09-22 19:05:27,150 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-55) Principal admin::internal synchronized >2014-09-22 19:05:28,694 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-6) Attempting to update VMs/Templates Ovf. >2014-09-22 20:05:27,160 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-74) Principal admin::internal synchronized >2014-09-22 20:05:28,716 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-65) Attempting to update VMs/Templates Ovf. >2014-09-22 21:05:27,171 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-61) Principal admin::internal synchronized >2014-09-22 21:05:28,734 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-09-22 22:05:27,182 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-22 22:05:28,754 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-51) Attempting to update VMs/Templates Ovf. >2014-09-22 23:05:27,194 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-22 23:05:28,774 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-68) Attempting to update VMs/Templates Ovf. >2014-09-23 00:05:27,204 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-40) Principal admin::internal synchronized >2014-09-23 00:05:28,793 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-55) Attempting to update VMs/Templates Ovf. >2014-09-23 01:05:27,214 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-15) Principal admin::internal synchronized >2014-09-23 01:05:28,816 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-90) Attempting to update VMs/Templates Ovf. >2014-09-23 02:05:27,223 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-54) Principal admin::internal synchronized >2014-09-23 02:05:28,835 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-83) Attempting to update VMs/Templates Ovf. >2014-09-23 03:05:27,236 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-50) Principal admin::internal synchronized >2014-09-23 03:05:28,858 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf. >2014-09-23 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-25) Start deleteAgedOutAuditLogs >2014-09-23 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-18) Start deleteAgedOutCommandEntities >2014-09-23 03:35:35,004 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-25) Finished deleteAgedOutAuditLogs >2014-09-23 03:35:35,006 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-18) Finished deleteAgedOutCommandEntities >2014-09-23 04:05:27,249 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-16) Principal admin::internal synchronized >2014-09-23 04:05:28,880 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-09-23 05:05:27,259 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-23 05:05:28,899 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-54) Attempting to update VMs/Templates Ovf. >2014-09-23 06:05:27,270 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-97) Principal admin::internal synchronized >2014-09-23 06:05:28,919 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-3) Attempting to update VMs/Templates Ovf. >2014-09-23 07:05:27,280 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-23 07:05:28,937 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-09-23 08:05:27,290 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-82) Principal admin::internal synchronized >2014-09-23 08:05:28,959 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-34) Attempting to update VMs/Templates Ovf. >2014-09-23 09:05:27,300 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-71) Principal admin::internal synchronized >2014-09-23 09:05:28,981 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-60) Attempting to update VMs/Templates Ovf. >2014-09-23 10:05:27,310 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-4) Principal admin::internal synchronized >2014-09-23 10:05:29,001 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-23 11:05:27,320 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-27) Principal admin::internal synchronized >2014-09-23 11:05:29,021 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-09-23 12:05:27,332 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-23 12:05:29,040 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-14) Attempting to update VMs/Templates Ovf. >2014-09-23 13:05:27,340 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-53) Principal admin::internal synchronized >2014-09-23 13:05:29,069 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-87) Attempting to update VMs/Templates Ovf. >2014-09-23 13:52:57,474 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-23 13:52:57,475 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-23 13:52:59,551 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-23 13:52:59,551 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-11) Invalid locale found in configuration: >2014-09-23 13:53:05,572 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-11) Running command: LoginAdminUserCommand internal: false. >2014-09-23 13:53:05,583 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-23 13:53:05,665 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-5) Running command: LoginUserCommand internal: false. >2014-09-23 13:54:21,531 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-9) [265c3d20] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-23 13:54:21,584 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-23 13:54:21,589 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] START, SetVdsStatusVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 2f8bad8a >2014-09-23 13:54:21,590 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] VDS rack is spm and moved from up calling resetIrs. >2014-09-23 13:54:21,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, ignoreStopFailed = false), log id: 76ba71e7 >2014-09-23 13:54:21,594 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] START, SpmStopVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 4cb4ee70 >2014-09-23 13:54:21,600 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-23 13:54:22,778 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] FINISH, SpmStopVDSCommand, log id: 4cb4ee70 >2014-09-23 13:54:22,788 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] FINISH, ResetIrsVDSCommand, log id: 76ba71e7 >2014-09-23 13:54:22,802 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] FINISH, SetVdsStatusVDSCommand, log id: 2f8bad8a >2014-09-23 13:54:22,823 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-19) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = e58f9fdb-4a02-44ea-be4d-e2162edb495c : rack >2014-09-23 13:54:22,835 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-23 13:54:22,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-10) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-23 13:54:22,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-10) Removing vds [e58f9fdb-4a02-44ea-be4d-e2162edb495c] from the domain in maintenance cache >2014-09-23 13:54:22,862 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-10) Removing host(s) [e58f9fdb-4a02-44ea-be4d-e2162edb495c] from hosts unseen domain report cache >2014-09-23 13:54:22,869 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-19) START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 44dc17b3 >2014-09-23 13:54:22,875 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-19) [265c3d20] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-23 13:54:22,885 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-19) [265c3d20] Correlation ID: 265c3d20, Job ID: d0850036-e118-4a9f-b0e9-2cfbce9faeb8, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-23 13:54:25,530 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-19) FINISH, DisconnectStoragePoolVDSCommand, log id: 44dc17b3 >2014-09-23 13:54:25,531 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-19) [3db454f1] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-23 13:54:25,545 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-19) [3db454f1] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 584338e4 >2014-09-23 13:54:25,815 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-19) [3db454f1] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 584338e4 >2014-09-23 13:54:30,120 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-59) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : no reporting hosts >2014-09-23 13:54:30,124 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-59) Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : no reporting hosts >2014-09-23 13:54:30,131 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-59) [62f22e94] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-23 13:54:30,150 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-59) [62f22e94] Correlation ID: 62f22e94, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-23 13:54:57,569 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-4) [7b28b1a8] Lock Acquired to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-23 13:54:57,604 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Running command: ActivateVdsCommand internal: false. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-23 13:54:57,605 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Before acquiring lock in order to prevent monitoring for host rack from data-center Default >2014-09-23 13:54:57,605 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Lock acquired, from now a monitoring of host will be skipped for host rack from data-center Default >2014-09-23 13:54:57,618 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] START, SetVdsStatusVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 3a318367 >2014-09-23 13:54:57,631 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] FINISH, SetVdsStatusVDSCommand, log id: 3a318367 >2014-09-23 13:54:57,680 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Activate finished. Lock released. Monitoring can run now for host rack from data-center Default >2014-09-23 13:54:57,697 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Correlation ID: 7b28b1a8, Job ID: 4d9303dd-7f27-4d1d-b3c3-15013ab9dc50, Call Stack: null, Custom Event ID: -1, Message: Host rack was activated by admin. >2014-09-23 13:54:57,705 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-35) [7b28b1a8] Lock freed to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-23 13:54:58,926 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) START, GetHardwareInfoVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vds=Host[rack,e58f9fdb-4a02-44ea-be4d-e2162edb495c]), log id: 388b5e60 >2014-09-23 13:54:58,931 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-47) FINISH, GetHardwareInfoVDSCommand, log id: 388b5e60 >2014-09-23 13:54:58,995 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-47) [34484291] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-23 13:54:59,016 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-47) [58d64577] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-23 13:54:59,028 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-23 13:54:59,041 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 707a1ce >2014-09-23 13:54:59,336 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 707a1ce >2014-09-23 13:54:59,336 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] Host rack storage connection was succeeded >2014-09-23 13:54:59,346 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-44) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vdsId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 5411ea7f >2014-09-23 13:54:59,861 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-44) FINISH, ConnectStoragePoolVDSCommand, log id: 5411ea7f >2014-09-23 13:54:59,875 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] START, SetMOMPolicyParametersVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 1c5b491f >2014-09-23 13:54:59,948 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] FINISH, SetMOMPolicyParametersVDSCommand, log id: 1c5b491f >2014-09-23 13:54:59,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host rack was set to Up. >2014-09-23 13:54:59,971 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host rack. >2014-09-23 13:54:59,980 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-47) [67dc5f9f] Correlation ID: 58d64577, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-23 13:55:00,026 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-47) [3f2548c3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-23 13:55:00,253 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - rack, spmStatus Free, storage pool Default >2014-09-23 13:55:00,260 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) starting spm on vds rack, storage pool Default, prevId -1, LVER 5 >2014-09-23 13:55:00,262 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-4) START, SpmStartVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=-1, prevLVER=5, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 6520edf1 >2014-09-23 13:55:00,270 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-4) spmStart polling started: taskId = 170265a5-8787-4e82-a34c-627416c9b676 >2014-09-23 13:55:20,552 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-7) [25362a9c] Lock Acquired to object EngineLock [exclusiveLocks= key: 021e5bea-9f8e-446d-b19c-fb68839d4972 value: DISK >key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-23 13:55:20,561 WARN [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-7) [25362a9c] CanDoAction of action UpdateVmDisk failed. Reasons:VAR__ACTION__UPDATE,VAR__TYPE__VM_DISK,ACTION_TYPE_FAILED_STORAGE_DOMAIN_STATUS_ILLEGAL2,$status Unknown >2014-09-23 13:55:20,563 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-7) [25362a9c] Lock freed to object EngineLock [exclusiveLocks= key: 021e5bea-9f8e-446d-b19c-fb68839d4972 value: DISK >key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-23 13:55:21,367 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-4) spmStart polling ended: taskId = 170265a5-8787-4e82-a34c-627416c9b676 task status = finished >2014-09-23 13:55:21,371 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-4) spmStart polling ended, spm status: SPM >2014-09-23 13:55:21,373 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-4) START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=170265a5-8787-4e82-a34c-627416c9b676), log id: 3e04fe95 >2014-09-23 13:55:21,377 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, HSMClearTaskVDSCommand, log id: 3e04fe95 >2014-09-23 13:55:21,377 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-4) FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@1588bd, log id: 6520edf1 >2014-09-23 13:55:21,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) Initialize Irs proxy from vds: dell-r210ii-16.rhev.lab.eng.brq.redhat.com >2014-09-23 13:55:21,391 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-4) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host rack (Address: dell-r210ii-16.rhev.lab.eng.brq.redhat.com). >2014-09-23 13:55:21,394 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to dell-r210ii-16.rhev.lab.eng.brq.redhat.com/10.34.62.208 >2014-09-23 13:55:21,400 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-50) START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1cbdef84 >2014-09-23 13:55:21,518 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-50) -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-23 13:55:21,520 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-50) START, HSMGetAllTasksInfoVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 5ac1aab5 >2014-09-23 13:55:21,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-50) FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 5ac1aab5 >2014-09-23 13:55:21,526 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-50) FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 1cbdef84 >2014-09-23 13:55:21,526 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-50) Discovered no tasks on Storage Pool Default >2014-09-23 13:55:52,641 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-4) [65e5ba72] Lock Acquired to object EngineLock [exclusiveLocks= key: 72c1d2e5-3906-472a-884a-beaf8178a948 value: DISK >key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM_DISK_BOOT >, sharedLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >] >2014-09-23 13:55:52,742 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-4) [65e5ba72] Running command: UpdateVmDiskCommand internal: false. Entities affected : ID: 72c1d2e5-3906-472a-884a-beaf8178a948 Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER >2014-09-23 13:55:52,828 INFO [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (ajp-/127.0.0.1:8702-4) [52f1fb64] Running command: ExtendImageSizeCommand internal: true. Entities affected : ID: 72c1d2e5-3906-472a-884a-beaf8178a948 Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER >2014-09-23 13:55:52,830 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendImageSizeVDSCommand] (ajp-/127.0.0.1:8702-4) [52f1fb64] START, ExtendImageSizeVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7a22a5fa >2014-09-23 13:55:53,674 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendImageSizeVDSCommand] (ajp-/127.0.0.1:8702-4) [52f1fb64] FINISH, ExtendImageSizeVDSCommand, log id: 7a22a5fa >2014-09-23 13:55:53,677 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-4) [52f1fb64] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 23a03b33-8917-40dd-85db-ec948143253a >2014-09-23 13:55:53,678 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-4) [52f1fb64] CommandMultiAsyncTasks::AttachTask: Attaching task eb42a5ed-9ad4-4445-b062-3318f85e5790 to command 23a03b33-8917-40dd-85db-ec948143253a. >2014-09-23 13:55:53,877 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-4) [52f1fb64] Adding task eb42a5ed-9ad4-4445-b062-3318f85e5790 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-23 13:55:53,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-4) [52f1fb64] Correlation ID: 65e5ba72, Call Stack: null, Custom Event ID: -1, Message: VM f20 f20_Disk1 disk was updated by admin. >2014-09-23 13:55:53,978 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-4) [52f1fb64] BaseAsyncTask::startPollingTask: Starting to poll task eb42a5ed-9ad4-4445-b062-3318f85e5790. >2014-09-23 13:56:01,913 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-93) Polling and updating Async Tasks: 1 tasks, 1 tasks to poll now >2014-09-23 13:56:01,920 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-93) SPMAsyncTask::PollTask: Polling task eb42a5ed-9ad4-4445-b062-3318f85e5790 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-23 13:56:01,921 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-93) BaseAsyncTask::onTaskEndSuccess: Task eb42a5ed-9ad4-4445-b062-3318f85e5790 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-23 13:56:01,921 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-93) CommandAsyncTask::endActionIfNecessary: All tasks of command 23a03b33-8917-40dd-85db-ec948143253a has ended -> executing endAction >2014-09-23 13:56:01,922 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-93) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 23a03b33-8917-40dd-85db-ec948143253a): calling endAction . >2014-09-23 13:56:01,923 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-17) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction UpdateVmDisk, executionIndex: 0 >2014-09-23 13:56:02,002 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (org.ovirt.thread.pool-6-thread-17) [65e5ba72] Ending command successfully: org.ovirt.engine.core.bll.UpdateVmDiskCommand >2014-09-23 13:56:02,003 INFO [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (org.ovirt.thread.pool-6-thread-17) Ending command successfully: org.ovirt.engine.core.bll.ExtendImageSizeCommand >2014-09-23 13:56:02,093 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, ExtendVmDiskSizeVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vmId=00000000-0000-0000-0000-000000000000), log id: 77f30f83 >2014-09-23 13:56:02,100 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-17) Failed in ExtendVmDiskSizeVDS method >2014-09-23 13:56:02,101 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-17) Command ExtendVmDiskSizeVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vmId=00000000-0000-0000-0000-000000000000) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to ExtendVmDiskSizeVDS, error = Virtual machine does not exist, code = 1 >2014-09-23 13:56:02,101 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, ExtendVmDiskSizeVDSCommand, log id: 77f30f83 >2014-09-23 13:56:02,102 WARN [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (org.ovirt.thread.pool-6-thread-17) Failed to update VM f20 with the new volume size due to error: VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to ExtendVmDiskSizeVDS, error = Virtual machine does not exist, code = 1 (Failed with error noVM and code 1).VM should be restarted to detect the new size. >2014-09-23 13:56:02,136 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to update VM 'f20' with the new volume size. VM restart is recommended. >2014-09-23 13:56:02,139 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 72c1d2e5-3906-472a-884a-beaf8178a948, imageId = 455469db-f6b6-4e6a-b906-8dc04ce3907a), log id: 793143a1 >2014-09-23 13:56:02,157 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@f8771042, log id: 793143a1 >2014-09-23 13:56:02,178 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-17) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Size of the disk 'f20_Disk1' was successfully updated to 30 GB by admin. >2014-09-23 13:56:02,278 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (org.ovirt.thread.pool-6-thread-17) Lock freed to object EngineLock [exclusiveLocks= key: 72c1d2e5-3906-472a-884a-beaf8178a948 value: DISK >key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM_DISK_BOOT >, sharedLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >] >2014-09-23 13:56:02,318 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-17) Correlation ID: 65e5ba72, Call Stack: null, Custom Event ID: -1, Message: VM f20 f20_Disk1 disk was updated by admin. >2014-09-23 13:56:02,319 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-17) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type UpdateVmDisk completed, handling the result. >2014-09-23 13:56:02,319 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-17) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type UpdateVmDisk succeeded, clearing tasks. >2014-09-23 13:56:02,319 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-17) SPMAsyncTask::ClearAsyncTask: Attempting to clear task eb42a5ed-9ad4-4445-b062-3318f85e5790 >2014-09-23 13:56:02,321 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = eb42a5ed-9ad4-4445-b062-3318f85e5790), log id: fc0d432 >2014-09-23 13:56:02,323 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=eb42a5ed-9ad4-4445-b062-3318f85e5790), log id: 3cab26b2 >2014-09-23 13:56:02,473 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, HSMClearTaskVDSCommand, log id: 3cab26b2 >2014-09-23 13:56:02,473 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, SPMClearTaskVDSCommand, log id: fc0d432 >2014-09-23 13:56:02,739 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-17) BaseAsyncTask::removeTaskFromDB: Removed task eb42a5ed-9ad4-4445-b062-3318f85e5790 from DataBase >2014-09-23 13:56:02,740 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-17) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 23a03b33-8917-40dd-85db-ec948143253a >2014-09-23 13:56:16,082 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-7) [3606bf56] Lock Acquired to object EngineLock [exclusiveLocks= key: 021e5bea-9f8e-446d-b19c-fb68839d4972 value: DISK >key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-23 13:56:16,097 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (ajp-/127.0.0.1:8702-7) [3606bf56] Running command: UpdateVmDiskCommand internal: false. Entities affected : ID: 021e5bea-9f8e-446d-b19c-fb68839d4972 Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER >2014-09-23 13:56:16,124 INFO [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (ajp-/127.0.0.1:8702-7) [24490c6a] Running command: ExtendImageSizeCommand internal: true. Entities affected : ID: 021e5bea-9f8e-446d-b19c-fb68839d4972 Type: DiskAction group EDIT_DISK_PROPERTIES with role type USER >2014-09-23 13:56:16,125 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendImageSizeVDSCommand] (ajp-/127.0.0.1:8702-7) [24490c6a] START, ExtendImageSizeVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 615e0d08 >2014-09-23 13:56:16,674 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ExtendImageSizeVDSCommand] (ajp-/127.0.0.1:8702-7) [24490c6a] FINISH, ExtendImageSizeVDSCommand, log id: 615e0d08 >2014-09-23 13:56:16,675 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (ajp-/127.0.0.1:8702-7) [24490c6a] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 03278790-57cb-44e5-b5bb-db47797eb9d4 >2014-09-23 13:56:16,675 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (ajp-/127.0.0.1:8702-7) [24490c6a] CommandMultiAsyncTasks::AttachTask: Attaching task 12c191bc-111a-4cd9-9e01-af22b29da298 to command 03278790-57cb-44e5-b5bb-db47797eb9d4. >2014-09-23 13:56:16,709 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ajp-/127.0.0.1:8702-7) [24490c6a] Adding task 12c191bc-111a-4cd9-9e01-af22b29da298 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-23 13:56:16,773 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [24490c6a] Correlation ID: 3606bf56, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 el6.5_Disk1 disk was updated by admin. >2014-09-23 13:56:16,773 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (ajp-/127.0.0.1:8702-7) [24490c6a] BaseAsyncTask::startPollingTask: Starting to poll task 12c191bc-111a-4cd9-9e01-af22b29da298. >2014-09-23 13:56:21,926 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-33) Polling and updating Async Tasks: 2 tasks, 1 tasks to poll now >2014-09-23 13:56:21,931 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-33) SPMAsyncTask::PollTask: Polling task 12c191bc-111a-4cd9-9e01-af22b29da298 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-23 13:56:21,931 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-33) BaseAsyncTask::onTaskEndSuccess: Task 12c191bc-111a-4cd9-9e01-af22b29da298 (Parent Command UpdateVmDisk, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-23 13:56:21,932 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-33) CommandAsyncTask::endActionIfNecessary: All tasks of command 03278790-57cb-44e5-b5bb-db47797eb9d4 has ended -> executing endAction >2014-09-23 13:56:21,932 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-33) CommandAsyncTask::endAction: Ending action for 1 tasks (command ID: 03278790-57cb-44e5-b5bb-db47797eb9d4): calling endAction . >2014-09-23 13:56:21,932 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-27) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction UpdateVmDisk, executionIndex: 0 >2014-09-23 13:56:21,944 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (org.ovirt.thread.pool-6-thread-27) [3606bf56] Ending command successfully: org.ovirt.engine.core.bll.UpdateVmDiskCommand >2014-09-23 13:56:21,946 INFO [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (org.ovirt.thread.pool-6-thread-27) Ending command successfully: org.ovirt.engine.core.bll.ExtendImageSizeCommand >2014-09-23 13:56:21,981 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-27) START, ExtendVmDiskSizeVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vmId=00000000-0000-0000-0000-000000000000), log id: 3d3c35a5 >2014-09-23 13:56:21,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-27) Failed in ExtendVmDiskSizeVDS method >2014-09-23 13:56:21,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-27) Command ExtendVmDiskSizeVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vmId=00000000-0000-0000-0000-000000000000) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to ExtendVmDiskSizeVDS, error = Virtual machine does not exist, code = 1 >2014-09-23 13:56:21,986 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ExtendVmDiskSizeVDSCommand] (org.ovirt.thread.pool-6-thread-27) FINISH, ExtendVmDiskSizeVDSCommand, log id: 3d3c35a5 >2014-09-23 13:56:21,986 WARN [org.ovirt.engine.core.bll.ExtendImageSizeCommand] (org.ovirt.thread.pool-6-thread-27) Failed to update VM el6.5 with the new volume size due to error: VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to ExtendVmDiskSizeVDS, error = Virtual machine does not exist, code = 1 (Failed with error noVM and code 1).VM should be restarted to detect the new size. >2014-09-23 13:56:21,997 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to update VM 'el6.5' with the new volume size. VM restart is recommended. >2014-09-23 13:56:21,998 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-27) START, GetImageInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 021e5bea-9f8e-446d-b19c-fb68839d4972, imageId = 7172d947-e2c8-4595-80bf-933c22f83b06), log id: 48084921 >2014-09-23 13:56:22,156 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (org.ovirt.thread.pool-6-thread-27) FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.DiskImage@870d808f, log id: 48084921 >2014-09-23 13:56:22,169 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Size of the disk 'el6.5_Disk1' was successfully updated to 30 GB by admin. >2014-09-23 13:56:22,279 INFO [org.ovirt.engine.core.bll.UpdateVmDiskCommand] (org.ovirt.thread.pool-6-thread-27) Lock freed to object EngineLock [exclusiveLocks= key: 021e5bea-9f8e-446d-b19c-fb68839d4972 value: DISK >key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM_DISK_BOOT >, sharedLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >] >2014-09-23 13:56:22,302 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27) Correlation ID: 3606bf56, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 el6.5_Disk1 disk was updated by admin. >2014-09-23 13:56:22,302 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type UpdateVmDisk completed, handling the result. >2014-09-23 13:56:22,302 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type UpdateVmDisk succeeded, clearing tasks. >2014-09-23 13:56:22,302 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-27) SPMAsyncTask::ClearAsyncTask: Attempting to clear task 12c191bc-111a-4cd9-9e01-af22b29da298 >2014-09-23 13:56:22,303 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-27) START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 12c191bc-111a-4cd9-9e01-af22b29da298), log id: 10927818 >2014-09-23 13:56:22,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-27) START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=12c191bc-111a-4cd9-9e01-af22b29da298), log id: 2f568a1d >2014-09-23 13:56:22,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-27) FINISH, HSMClearTaskVDSCommand, log id: 2f568a1d >2014-09-23 13:56:22,440 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-27) FINISH, SPMClearTaskVDSCommand, log id: 10927818 >2014-09-23 13:56:22,473 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-27) BaseAsyncTask::removeTaskFromDB: Removed task 12c191bc-111a-4cd9-9e01-af22b29da298 from DataBase >2014-09-23 13:56:22,473 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-27) CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 03278790-57cb-44e5-b5bb-db47797eb9d4 >2014-09-23 13:57:07,060 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-5) Setting new tasks map. The map contains now 1 tasks >2014-09-23 13:57:37,061 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Setting new tasks map. The map contains now 0 tasks >2014-09-23 13:57:37,061 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-91) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-23 14:05:27,351 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) Principal admin::internal synchronized >2014-09-23 14:05:29,089 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) Attempting to update VMs/Templates Ovf. >2014-09-23 14:05:29,482 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] Lock Acquired to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-23 14:05:29,486 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] Running command: ProcessOvfUpdateForStorageDomainCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-23 14:05:29,555 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 5d0c2345 >2014-09-23 14:05:29,555 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-23 14:05:29,556 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 14:54:47 CEST 2014"} > >2014-09-23 14:05:29,714 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [37b24b70] FINISH, SetVolumeDescriptionVDSCommand, log id: 5d0c2345 >2014-09-23 14:05:29,822 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-23 14:05:29,834 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] START, UploadStreamVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 6f3d34b1 >2014-09-23 14:05:29,834 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- executeVdsBrokerCommand: >2014-09-23 14:05:29,834 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > size=20,480 > >2014-09-23 14:05:31,420 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] FINISH, UploadStreamVDSCommand, log id: 6f3d34b1 >2014-09-23 14:05:31,436 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-4) [392c1223] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5 >2014-09-23 14:05:31,436 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-4) [392c1223] CommandMultiAsyncTasks::AttachTask: Attaching task 841dddbd-2567-435e-8e72-4cd534c9988e to command 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5. >2014-09-23 14:05:31,458 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-4) [392c1223] Adding task 841dddbd-2567-435e-8e72-4cd534c9988e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-23 14:05:31,468 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-23 14:05:31,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = f7db4018-164f-4243-99a5-381211cde2e2, imageId = 285be822-e08e-424e-ac3a-f6cdfbd6d0f0), log id: 736bf3de >2014-09-23 14:05:31,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-23 14:05:31,478 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=f7db4018-164f-4243-99a5-381211cde2e2 > volUUID=285be822-e08e-424e-ac3a-f6cdfbd6d0f0 > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Tue Sep 23 14:05:29 CEST 2014","Size":20480} > >2014-09-23 14:05:31,785 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] FINISH, SetVolumeDescriptionVDSCommand, log id: 736bf3de >2014-09-23 14:05:31,794 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 660f3fc9 >2014-09-23 14:05:31,794 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-23 14:05:31,794 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":false,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Thu Sep 11 14:54:47 CEST 2014"} > >2014-09-23 14:05:31,977 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [392c1223] FINISH, SetVolumeDescriptionVDSCommand, log id: 660f3fc9 >2014-09-23 14:05:32,029 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Running command: UploadStreamCommand internal: true. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: Storage >2014-09-23 14:05:32,032 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] START, UploadStreamVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 59cd7fc >2014-09-23 14:05:32,032 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] -- executeVdsBrokerCommand: >2014-09-23 14:05:32,032 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] -- parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > size=20,480 > >2014-09-23 14:05:33,161 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] FINISH, UploadStreamVDSCommand, log id: 59cd7fc >2014-09-23 14:05:33,164 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-4) [1da6da9d] CommandMultiAsyncTasks::AttachTask: Attaching task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede to command 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5. >2014-09-23 14:05:33,195 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Adding task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters), polling hasn't started yet.. >2014-09-23 14:05:33,215 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-23 14:05:33,215 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-23 14:05:33,216 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-23 14:05:33,216 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-23 14:05:33,216 INFO [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-23 14:05:33,219 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] START, SetVolumeDescriptionVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageGroupId = 2d5eebeb-0696-425a-b54d-d751f8e6ae8a, imageId = 32717d52-27ce-439e-8803-74ee646b94ea), log id: 22087b03 >2014-09-23 14:05:33,219 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] -- executeIrsBrokerCommand: calling 'setVolumeDescription' >2014-09-23 14:05:33,220 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] -- setVolumeDescription parameters: > spUUID=00000002-0002-0002-0002-0000000001ad > sdUUID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc > imageGroupGUID=2d5eebeb-0696-425a-b54d-d751f8e6ae8a > volUUID=32717d52-27ce-439e-8803-74ee646b94ea > description={"Updated":true,"Disk Description":"OVF_STORE","Storage Domains":[{"uuid":"eb633636-3019-47a9-9f5a-fc81f8dfd7dc"}],"Last Updated":"Tue Sep 23 14:05:29 CEST 2014","Size":20480} > >2014-09-23 14:05:33,493 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (DefaultQuartzScheduler_Worker-4) [1da6da9d] FINISH, SetVolumeDescriptionVDSCommand, log id: 22087b03 >2014-09-23 14:05:33,499 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) [1da6da9d] BaseAsyncTask::startPollingTask: Starting to poll task 841dddbd-2567-435e-8e72-4cd534c9988e. >2014-09-23 14:05:33,499 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-4) [1da6da9d] BaseAsyncTask::startPollingTask: Starting to poll task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede. >2014-09-23 14:05:41,998 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-38) Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now >2014-09-23 14:05:42,004 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) SPMAsyncTask::PollTask: Polling task 841dddbd-2567-435e-8e72-4cd534c9988e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-23 14:05:42,004 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) BaseAsyncTask::onTaskEndSuccess: Task 841dddbd-2567-435e-8e72-4cd534c9988e (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-23 14:05:42,005 INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (DefaultQuartzScheduler_Worker-38) Task with DB Task ID 3879f436-4242-4d8f-b789-cc2780de8eea and VDSM Task ID df1ee0b0-f3ad-486b-9256-aa50dc4b3ede is in state Polling. End action for command 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5 will proceed when all the entitys tasks are completed. >2014-09-23 14:05:42,005 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) SPMAsyncTask::PollTask: Polling task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) returned status finished, result 'success'. >2014-09-23 14:05:42,005 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (DefaultQuartzScheduler_Worker-38) BaseAsyncTask::onTaskEndSuccess: Task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede (Parent Command ProcessOvfUpdateForStorageDomain, Parameters Type org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters) ended successfully. >2014-09-23 14:05:42,005 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-38) CommandAsyncTask::endActionIfNecessary: All tasks of command 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5 has ended -> executing endAction >2014-09-23 14:05:42,006 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (DefaultQuartzScheduler_Worker-38) CommandAsyncTask::endAction: Ending action for 2 tasks (command ID: 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5): calling endAction . >2014-09-23 14:05:42,006 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-12) CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction ProcessOvfUpdateForStorageDomain, executionIndex: 0 >2014-09-23 14:05:42,026 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Ending command successfully: org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand >2014-09-23 14:05:42,027 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Trying to release a shared lock for key: 00000002-0002-0002-0002-0000000001adOVF_UPDATE , but lock does not exist >2014-09-23 14:05:42,027 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Trying to release exclusive lock which does not exist, lock key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8aDISK >2014-09-23 14:05:42,028 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Trying to release exclusive lock which does not exist, lock key: eb633636-3019-47a9-9f5a-fc81f8dfd7dcSTORAGE >2014-09-23 14:05:42,028 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Trying to release exclusive lock which does not exist, lock key: f7db4018-164f-4243-99a5-381211cde2e2DISK >2014-09-23 14:05:42,028 INFO [org.ovirt.engine.core.bll.ProcessOvfUpdateForStorageDomainCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] Lock freed to object EngineLock [exclusiveLocks= key: 2d5eebeb-0696-425a-b54d-d751f8e6ae8a value: DISK >key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >key: f7db4018-164f-4243-99a5-381211cde2e2 value: DISK >, sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: OVF_UPDATE >] >2014-09-23 14:05:42,028 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain completed, handling the result. >2014-09-23 14:05:42,028 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type ProcessOvfUpdateForStorageDomain succeeded, clearing tasks. >2014-09-23 14:05:42,030 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] SPMAsyncTask::ClearAsyncTask: Attempting to clear task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede >2014-09-23 14:05:42,031 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = df1ee0b0-f3ad-486b-9256-aa50dc4b3ede), log id: d4b0aeb >2014-09-23 14:05:42,034 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=df1ee0b0-f3ad-486b-9256-aa50dc4b3ede), log id: 39b9d11e >2014-09-23 14:05:42,069 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] FINISH, HSMClearTaskVDSCommand, log id: 39b9d11e >2014-09-23 14:05:42,069 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] FINISH, SPMClearTaskVDSCommand, log id: d4b0aeb >2014-09-23 14:05:42,079 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] BaseAsyncTask::removeTaskFromDB: Removed task df1ee0b0-f3ad-486b-9256-aa50dc4b3ede from DataBase >2014-09-23 14:05:42,079 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 841dddbd-2567-435e-8e72-4cd534c9988e >2014-09-23 14:05:42,080 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] START, SPMClearTaskVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, taskId = 841dddbd-2567-435e-8e72-4cd534c9988e), log id: 2f2f971a >2014-09-23 14:05:42,084 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] START, HSMClearTaskVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, taskId=841dddbd-2567-435e-8e72-4cd534c9988e), log id: 5d694429 >2014-09-23 14:05:42,144 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] FINISH, HSMClearTaskVDSCommand, log id: 5d694429 >2014-09-23 14:05:42,144 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (org.ovirt.thread.pool-6-thread-12) [37b24b70] FINISH, SPMClearTaskVDSCommand, log id: 2f2f971a >2014-09-23 14:05:42,152 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] BaseAsyncTask::removeTaskFromDB: Removed task 841dddbd-2567-435e-8e72-4cd534c9988e from DataBase >2014-09-23 14:05:42,152 INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (org.ovirt.thread.pool-6-thread-12) [37b24b70] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 92d05239-c5ed-42d0-a3ff-aeecfabe3eb5 >2014-09-23 14:07:07,078 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) Setting new tasks map. The map contains now 0 tasks >2014-09-23 14:07:07,078 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (DefaultQuartzScheduler_Worker-88) Cleared all tasks of pool 00000002-0002-0002-0002-0000000001ad. >2014-09-23 15:05:27,363 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-17) Principal admin::internal synchronized >2014-09-23 15:05:33,500 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-96) Attempting to update VMs/Templates Ovf. >2014-09-23 15:13:49,697 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:13:49,764 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-41) [31182dee] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:13:49,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-41) [31182dee] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3083d331 >2014-09-23 15:15:59,885 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-41) [31182dee] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to connect Host rack to the Storage Domains ISO_DOMAIN. >2014-09-23 15:15:59,886 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-41) [31182dee] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=477}, log id: 3083d331 >2014-09-23 15:15:59,891 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-41) [31182dee] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: The error message for connection pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso returned by VDSM was: Problem while trying to mount target >2014-09-23 15:15:59,892 ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper] (org.ovirt.thread.pool-6-thread-41) [31182dee] The connection with details pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso failed because of error code 477 and error message is: problem while trying to mount target >2014-09-23 15:15:59,917 ERROR [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-41) [31182dee] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand. >2014-09-23 15:15:59,919 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] START, AttachStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = fc587b6a-bac3-4668-99de-7f6fd6acce7f), log id: 32510b62 >2014-09-23 15:16:00,476 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] Failed in AttachStorageDomainVDS method >2014-09-23 15:16:00,513 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] IrsBroker::Failed::AttachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Storage domain does not exist: (u'fc587b6a-bac3-4668-99de-7f6fd6acce7f',), code = 358 >2014-09-23 15:16:00,522 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] FINISH, AttachStorageDomainVDSCommand, log id: 32510b62 >2014-09-23 15:16:00,522 ERROR [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] Command org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to AttachStorageDomainVDS, error = Storage domain does not exist: (u'fc587b6a-bac3-4668-99de-7f6fd6acce7f',), code = 358 (Failed with error StorageDomainDoesNotExist and code 358) >2014-09-23 15:16:00,527 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-21) [3997980a] Command [id=54097a12-63d3-4be1-b313-ca779919c565]: Compensating NEW_ENTITY_ID of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageId = fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 15:16:00,559 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-21) [3997980a] Correlation ID: 3997980a, Job ID: bc452e05-ce16-4424-9edd-34fec2b9c7ff, Call Stack: null, Custom Event ID: -1, Message: Failed to attach Storage Domain ISO_DOMAIN to Data Center Default. (User: admin) >2014-09-23 15:20:27,837 INFO [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] Running command: AttachStorageDomainToPoolCommand internal: false. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:20:27,857 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-8) [5cb1418b] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:20:27,863 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-8) [5cb1418b] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2dd94fac >2014-09-23 15:20:33,195 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-8) [5cb1418b] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 2dd94fac >2014-09-23 15:20:33,198 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] START, AttachStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = fc587b6a-bac3-4668-99de-7f6fd6acce7f), log id: 73e68a4f >2014-09-23 15:20:47,503 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.AttachStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] FINISH, AttachStorageDomainVDSCommand, log id: 73e68a4f >2014-09-23 15:20:47,550 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] START, GetImagesListVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, sdUUID = fc587b6a-bac3-4668-99de-7f6fd6acce7f), log id: 3e2f685d >2014-09-23 15:20:47,588 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImagesListVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] FINISH, GetImagesListVDSCommand, return: [11111111-1111-1111-1111-111111111111], log id: 3e2f685d >2014-09-23 15:20:47,661 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] START, GetVolumesListVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = fc587b6a-bac3-4668-99de-7f6fd6acce7f, imageGroupId = 11111111-1111-1111-1111-111111111111), log id: 565afda4 >2014-09-23 15:20:47,672 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetVolumesListVDSCommand] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] FINISH, GetVolumesListVDSCommand, return: [], log id: 565afda4 >2014-09-23 15:20:47,672 ERROR [org.ovirt.engine.core.bll.GetUnregisteredDisksQuery] (org.ovirt.thread.pool-6-thread-14) [2c5b5147] Could not get populated disk, reason: null >2014-09-23 15:20:47,673 WARN [org.ovirt.engine.core.bll.storage.AttachStorageDomainToPoolCommand] (org.ovirt.thread.pool-6-thread-14) There are no OVF_STORE disks on storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-23 15:20:47,683 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] Lock Acquired to object EngineLock [exclusiveLocks= key: fc587b6a-bac3-4668-99de-7f6fd6acce7f value: STORAGE >, sharedLocks= ] >2014-09-23 15:20:47,706 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] Running command: ActivateStorageDomainCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:20:47,721 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] Lock freed to object EngineLock [exclusiveLocks= key: fc587b6a-bac3-4668-99de-7f6fd6acce7f value: STORAGE >, sharedLocks= ] >2014-09-23 15:20:47,721 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/23/14 3:20 PM >2014-09-23 15:20:47,729 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-6-thread-17) [79de0eaa] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-23 15:20:47,735 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-17) [79de0eaa] START, ConnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 56b645d7 >2014-09-23 15:20:47,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-6-thread-17) [79de0eaa] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 56b645d7 >2014-09-23 15:20:47,746 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = fc587b6a-bac3-4668-99de-7f6fd6acce7f), log id: 4aa066f3 >2014-09-23 15:20:47,971 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] FINISH, ActivateStorageDomainVDSCommand, return: true, log id: 4aa066f3 >2014-09-23 15:20:47,971 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] ActivateStorage Domain. After Connect all hosts to pool. Time:9/23/14 3:20 PM >2014-09-23 15:20:47,989 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-35) START, ConnectStoragePoolVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, vdsId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 599657ac >2014-09-23 15:20:48,287 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-35) FINISH, ConnectStoragePoolVDSCommand, log id: 599657ac >2014-09-23 15:20:48,287 INFO [org.ovirt.engine.core.bll.storage.RefreshPoolSingleAsyncOperation] (org.ovirt.thread.pool-6-thread-35) Refreshed vds rack in pool Default >2014-09-23 15:20:48,288 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-6-thread-14) [73ee625b] ActivateStorage Domain. After change storage pool status in vds. Time:9/23/14 3:20 PM >2014-09-23 15:20:48,292 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [73ee625b] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1301c4c1 >2014-09-23 15:20:48,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [73ee625b] Correlation ID: 73ee625b, Call Stack: null, Custom Event ID: -1, Message: Storage Domain ISO_DOMAIN (Data Center Default) was activated by <UNKNOWN> >2014-09-23 15:20:48,342 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [73ee625b] FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 1301c4c1 >2014-09-23 15:20:48,347 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-14) [73ee625b] Correlation ID: 2c5b5147, Job ID: 96daeaad-ed48-44f6-bca8-d50912ec05c6, Call Stack: null, Custom Event ID: -1, Message: Storage Domain ISO_DOMAIN was attached to Data Center Default by admin >2014-09-23 15:20:48,380 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [73ee625b] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: d2ecce >2014-09-23 15:20:48,402 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [73ee625b] FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: d2ecce >2014-09-23 15:47:01,175 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-23 15:47:01,175 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-23 16:05:27,373 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-89) Principal admin::internal synchronized >2014-09-23 16:05:33,525 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-09-23 17:05:27,385 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-23 17:05:33,548 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-63) Attempting to update VMs/Templates Ovf. >2014-09-23 17:10:26,000 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-7) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 36a0b6d1 >2014-09-23 17:10:26,002 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-49) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 3a18e22d >2014-09-23 17:10:26,021 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-7) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 36a0b6d1 >2014-09-23 17:10:26,029 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-7) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 17:10:26,038 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-49) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 3a18e22d >2014-09-23 17:10:26,046 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-49) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 18:05:27,396 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-99) Principal admin::internal synchronized >2014-09-23 18:05:33,575 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-09-23 18:10:26,059 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-5) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 32231be1 >2014-09-23 18:10:26,060 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-39) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: c0c8616 >2014-09-23 18:10:26,079 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-5) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 32231be1 >2014-09-23 18:10:26,088 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-5) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 18:10:26,097 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-39) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: c0c8616 >2014-09-23 18:10:26,104 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-39) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 19:05:27,407 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-63) Principal admin::internal synchronized >2014-09-23 19:05:33,610 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) Attempting to update VMs/Templates Ovf. >2014-09-23 19:10:26,114 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-2) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: bdad5b5 >2014-09-23 19:10:26,116 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4c046427 >2014-09-23 19:10:26,133 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-2) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: bdad5b5 >2014-09-23 19:10:26,142 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-2) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 19:10:26,150 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 4c046427 >2014-09-23 19:10:26,161 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-15) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 20:05:27,421 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-17) Principal admin::internal synchronized >2014-09-23 20:05:33,634 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-80) Attempting to update VMs/Templates Ovf. >2014-09-23 20:10:26,173 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 193c028d >2014-09-23 20:10:26,174 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4f421501 >2014-09-23 20:10:26,193 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 193c028d >2014-09-23 20:10:26,202 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-30) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 20:10:26,210 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 4f421501 >2014-09-23 20:10:26,218 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-34) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 21:05:27,435 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-23 21:05:33,661 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-38) Attempting to update VMs/Templates Ovf. >2014-09-23 21:10:26,228 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-31) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: c31c84d >2014-09-23 21:10:26,228 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 437a45f >2014-09-23 21:10:26,308 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-31) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: c31c84d >2014-09-23 21:10:26,316 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-31) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 21:10:26,372 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 437a45f >2014-09-23 21:10:26,382 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-19) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 22:05:27,446 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-96) Principal admin::internal synchronized >2014-09-23 22:05:33,689 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-86) Attempting to update VMs/Templates Ovf. >2014-09-23 22:10:26,394 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) [3775fd20] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 73c8a7b6 >2014-09-23 22:10:26,394 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-10) [3775fd20] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1743fc16 >2014-09-23 22:10:26,415 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) [3775fd20] FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 73c8a7b6 >2014-09-23 22:10:26,424 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-43) [3775fd20] Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 22:10:26,432 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-10) [3775fd20] FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 1743fc16 >2014-09-23 22:10:26,441 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-10) [3775fd20] Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 23:05:27,457 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-34) Principal admin::internal synchronized >2014-09-23 23:05:33,715 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-17) Attempting to update VMs/Templates Ovf. >2014-09-23 23:10:26,453 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-36) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 102885ef >2014-09-23 23:10:26,455 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-8) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 6c5164bc >2014-09-23 23:10:26,472 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-36) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 102885ef >2014-09-23 23:10:26,482 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-36) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-23 23:10:26,490 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-8) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 6c5164bc >2014-09-23 23:10:26,499 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-8) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 00:05:27,468 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-12) Principal admin::internal synchronized >2014-09-24 00:05:33,740 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-09-24 00:10:26,510 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-14) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 12fee8ee >2014-09-24 00:10:26,516 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-37) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2ce6bc2c >2014-09-24 00:10:26,531 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-14) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 12fee8ee >2014-09-24 00:10:26,541 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-14) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 00:10:26,549 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-37) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 2ce6bc2c >2014-09-24 00:10:26,555 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-37) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 01:05:27,480 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-56) Principal admin::internal synchronized >2014-09-24 01:05:33,765 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-09-24 01:10:26,564 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-40) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1e655522 >2014-09-24 01:10:26,564 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-32) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 611a630f >2014-09-24 01:10:26,585 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-40) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 1e655522 >2014-09-24 01:10:26,594 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-40) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 01:10:26,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-32) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 611a630f >2014-09-24 01:10:26,619 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-32) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 02:05:27,492 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-27) Principal admin::internal synchronized >2014-09-24 02:05:33,793 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-39) Attempting to update VMs/Templates Ovf. >2014-09-24 02:10:26,629 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 32e85e7 >2014-09-24 02:10:26,632 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5306e8d >2014-09-24 02:10:26,649 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 32e85e7 >2014-09-24 02:10:26,657 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-15) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 02:10:26,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 5306e8d >2014-09-24 02:10:26,675 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-4) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 03:05:27,504 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-95) Principal admin::internal synchronized >2014-09-24 03:05:33,818 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Attempting to update VMs/Templates Ovf. >2014-09-24 03:10:26,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-16) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1faeec5 >2014-09-24 03:10:26,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 151c887 >2014-09-24 03:10:26,706 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-16) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 1faeec5 >2014-09-24 03:10:26,717 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-16) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 03:10:26,724 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-28) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 151c887 >2014-09-24 03:10:26,732 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-28) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-39) Start deleteAgedOutAuditLogs >2014-09-24 03:35:35,002 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-52) Start deleteAgedOutCommandEntities >2014-09-24 03:35:35,005 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-39) Finished deleteAgedOutAuditLogs >2014-09-24 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-52) Finished deleteAgedOutCommandEntities >2014-09-24 04:05:27,514 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-16) Principal admin::internal synchronized >2014-09-24 04:05:33,846 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) Attempting to update VMs/Templates Ovf. >2014-09-24 04:10:26,744 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 6e3ff3ef >2014-09-24 04:10:26,744 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 28a80875 >2014-09-24 04:10:26,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 6e3ff3ef >2014-09-24 04:10:26,774 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-35) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 04:10:26,781 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 28a80875 >2014-09-24 04:10:26,792 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-19) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 05:05:27,525 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-93) Principal admin::internal synchronized >2014-09-24 05:05:33,873 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-13) Attempting to update VMs/Templates Ovf. >2014-09-24 05:10:26,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-38) [adb8dca] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 3a4d6169 >2014-09-24 05:10:26,805 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 705bbe3d >2014-09-24 05:10:26,824 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-38) [adb8dca] FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 3a4d6169 >2014-09-24 05:10:26,836 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-38) [adb8dca] Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 05:10:26,844 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-6) [adb8dca] FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 705bbe3d >2014-09-24 05:10:26,855 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-6) [adb8dca] Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 06:05:27,538 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-24 06:05:33,897 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-24) Attempting to update VMs/Templates Ovf. >2014-09-24 06:10:26,869 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4df8a48c >2014-09-24 06:10:26,870 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-22) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 13a433f8 >2014-09-24 06:10:26,888 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 4df8a48c >2014-09-24 06:10:26,900 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-48) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 06:10:26,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-22) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 13a433f8 >2014-09-24 06:10:26,914 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-22) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 07:05:27,549 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-24 07:05:33,933 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-5) Attempting to update VMs/Templates Ovf. >2014-09-24 07:10:26,929 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5cc14a71 >2014-09-24 07:10:26,930 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-47) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 737e158d >2014-09-24 07:10:26,953 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 5cc14a71 >2014-09-24 07:10:26,965 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-13) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 07:10:26,969 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-47) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 737e158d >2014-09-24 07:10:26,978 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-47) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 08:05:27,561 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-24 08:05:33,959 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) Attempting to update VMs/Templates Ovf. >2014-09-24 08:10:26,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-49) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5a782cad >2014-09-24 08:10:26,991 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 19c5b106 >2014-09-24 08:10:27,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-49) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 5a782cad >2014-09-24 08:10:27,024 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-49) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 08:10:27,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 19c5b106 >2014-09-24 08:10:27,059 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-26) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 09:05:27,572 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-76) Principal admin::internal synchronized >2014-09-24 09:05:33,985 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-26) Attempting to update VMs/Templates Ovf. >2014-09-24 09:10:27,070 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-1) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2c19bb9a >2014-09-24 09:10:27,072 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 514386c1 >2014-09-24 09:10:27,090 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-1) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 2c19bb9a >2014-09-24 09:10:27,100 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-1) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 09:10:27,110 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: 514386c1 >2014-09-24 09:10:27,118 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-17) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 10:04:47,278 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-24 10:04:47,278 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-24 10:04:58,300 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-12) Invalid locale found in configuration: >2014-09-24 10:04:58,300 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-12) Invalid locale found in configuration: >2014-09-24 10:05:07,764 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-12) Running command: LoginAdminUserCommand internal: false. >2014-09-24 10:05:07,776 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-12) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-24 10:05:07,880 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-6) Running command: LoginUserCommand internal: false. >2014-09-24 10:05:27,581 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-29) [3775fd20] Principal admin::internal synchronized >2014-09-24 10:05:34,009 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-43) Attempting to update VMs/Templates Ovf. >2014-09-24 10:10:27,129 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5de4fd5f >2014-09-24 10:10:27,129 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: fbdd22f >2014-09-24 10:10:27,149 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410270000.0, size=350183424}, virtio-win.iso={status=0, ctime=1410270000.0, size=149004288}}, log id: 5de4fd5f >2014-09-24 10:10:27,163 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-33) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 10:10:27,166 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410270000.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410270000.0, size=2949120}}, log id: fbdd22f >2014-09-24 10:10:27,186 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-35) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 10:23:15,472 WARN [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [169a82af] Failed to initiate vdsm-id request on host with message Command returned failure code 127 during SSH session 'root@slot-7.rhev.lab.eng.brq.redhat.com' >2014-09-24 10:23:15,499 INFO [org.ovirt.engine.core.bll.AddVdsCommand] (ajp-/127.0.0.1:8702-6) [169a82af] Running command: AddVdsCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group CREATE_HOST with role type ADMIN >2014-09-24 10:23:15,575 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [760caaa] Running command: AddVdsSpmIdCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:23:15,587 INFO [org.ovirt.engine.core.bll.AddVdsSpmIdCommand] (ajp-/127.0.0.1:8702-6) [760caaa] Lock freed to object EngineLock [exclusiveLocks= key: 00000002-0002-0002-0002-0000000001ad value: REGISTER_VDS >, sharedLocks= ] >2014-09-24 10:23:15,589 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] START, RemoveVdsVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 236d396f >2014-09-24 10:23:15,589 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] FINISH, RemoveVdsVDSCommand, log id: 236d396f >2014-09-24 10:23:15,591 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] START, AddVdsVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 40573a0d >2014-09-24 10:23:15,591 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] AddVds - entered , starting logic to add VDS 96620dcc-2509-48fe-841b-85247771c4fb >2014-09-24 10:23:15,593 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] AddVds - VDS 96620dcc-2509-48fe-841b-85247771c4fb was added, will try to add it to the resource manager >2014-09-24 10:23:15,593 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [760caaa] Entered VdsManager constructor >2014-09-24 10:23:15,616 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ajp-/127.0.0.1:8702-6) [760caaa] Initialize vdsBroker (slot-7.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-24 10:23:15,624 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ajp-/127.0.0.1:8702-6) [760caaa] VDS 96620dcc-2509-48fe-841b-85247771c4fb was added to the Resource Manager >2014-09-24 10:23:15,624 INFO [org.ovirt.engine.core.vdsbroker.AddVdsVDSCommand] (ajp-/127.0.0.1:8702-6) [760caaa] FINISH, AddVdsVDSCommand, log id: 40573a0d >2014-09-24 10:23:15,646 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [760caaa] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host blade. >2014-09-24 10:23:15,688 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Lock Acquired to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:23:15,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-6) [760caaa] Correlation ID: 169a82af, Job ID: c441266f-f8f2-4266-a5e9-7a2a2de934b7, Call Stack: null, Custom Event ID: -1, Message: Host blade was added by admin. >2014-09-24 10:23:15,703 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Running command: InstallVdsInternalCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:23:15,704 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Before Installation host 96620dcc-2509-48fe-841b-85247771c4fb, blade >2014-09-24 10:23:15,709 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=Installing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4ce0c1d7 >2014-09-24 10:23:15,723 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] FINISH, SetVdsStatusVDSCommand, log id: 4ce0c1d7 >2014-09-24 10:23:15,778 INFO [org.ovirt.engine.core.bll.InstallerMessages] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Installation slot-7.rhev.lab.eng.brq.redhat.com: Connected to host slot-7.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 4c:1b:40:24:df:34:ed:ff:72:b5:38:d6:63:7e:3d:f4 >2014-09-24 10:23:15,789 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Connected to host slot-7.rhev.lab.eng.brq.redhat.com with SSH key fingerprint: 4c:1b:40:24:df:34:ed:ff:72:b5:38:d6:63:7e:3d:f4. >2014-09-24 10:23:15,854 INFO [org.ovirt.engine.core.bll.VdsDeploy] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Installation of slot-7.rhev.lab.eng.brq.redhat.com. Executing command via SSH umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True < /var/cache/ovirt-engine/ovirt-host-deploy.tar >2014-09-24 10:23:15,868 INFO [org.ovirt.engine.core.uutils.ssh.SSHDialog] (org.ovirt.thread.pool-6-thread-20) SSH execute root@slot-7.rhev.lab.eng.brq.redhat.com 'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; rm -fr "${MYTMP}" && mkdir "${MYTMP}" && tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/setup DIALOG/dialect=str:machine DIALOG/customization=bool:True' >2014-09-24 10:23:18,189 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Initializing >2014-09-24 10:23:18,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Initializing. >2014-09-24 10:23:18,225 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Environment setup >2014-09-24 10:23:18,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Environment setup. >2014-09-24 10:23:18,297 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Environment packages setup >2014-09-24 10:23:18,309 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Environment packages setup. >2014-09-24 10:23:18,628 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-81) Initializing Host: blade >2014-09-24 10:23:34,207 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Programs detection >2014-09-24 10:23:34,221 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Programs detection. >2014-09-24 10:23:34,243 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Environment customization >2014-09-24 10:23:34,253 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Environment customization. >2014-09-24 10:23:34,764 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Kdump unsupported >2014-09-24 10:23:34,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Kdump unsupported. >2014-09-24 10:23:34,808 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Logs at host located at: '/tmp/ovirt-host-deploy-20140924102315-r3g6ne.log' >2014-09-24 10:23:34,820 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Logs at host located at: '/tmp/ovirt-host-deploy-20140924102315-r3g6ne.log'. >2014-09-24 10:23:34,928 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Host slot-7.rhev.lab.eng.brq.redhat.com reports unique id 4C4C4544-0052-4C10-8058-B5C04F43354A >2014-09-24 10:23:34,957 INFO [org.ovirt.engine.core.bll.VdsDeploy] (VdsDeploy) Assigning unique id 4C4C4544-0052-4C10-8058-B5C04F43354A to Host slot-7.rhev.lab.eng.brq.redhat.com >2014-09-24 10:23:35,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Disabling Kdump integration >2014-09-24 10:23:35,103 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Disabling Kdump integration. >2014-09-24 10:23:35,137 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Setup validation >2014-09-24 10:23:35,147 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Setup validation. >2014-09-24 10:23:35,272 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Hardware supports virtualization >2014-09-24 10:23:35,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Hardware supports virtualization. >2014-09-24 10:23:35,536 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Transaction setup >2014-09-24 10:23:35,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Transaction setup. >2014-09-24 10:23:35,622 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-24 10:23:35,632 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Misc configuration. >2014-09-24 10:23:35,633 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Package installation >2014-09-24 10:23:35,645 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Package installation. >2014-09-24 10:23:56,996 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Status: Downloading Packages >2014-09-24 10:23:57,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Status: Downloading Packages. >2014-09-24 10:23:57,112 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: OpenIPMI-modalias-2.0.19-11.el7.x86_64 >2014-09-24 10:23:57,124 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: OpenIPMI-modalias-2.0.19-11.el7.x86_64. >2014-09-24 10:23:57,138 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: attr-2.4.46-12.el7.x86_64 >2014-09-24 10:23:57,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: attr-2.4.46-12.el7.x86_64. >2014-09-24 10:23:57,148 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: audit-libs-python-2.3.3-4.el7.x86_64 >2014-09-24 10:23:57,158 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: audit-libs-python-2.3.3-4.el7.x86_64. >2014-09-24 10:23:57,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: augeas-1.1.0-12.el7.x86_64 >2014-09-24 10:23:57,202 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: augeas-1.1.0-12.el7.x86_64. >2014-09-24 10:23:57,224 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: augeas-libs-1.1.0-12.el7.x86_64 >2014-09-24 10:23:57,236 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: augeas-libs-1.1.0-12.el7.x86_64. >2014-09-24 10:23:57,260 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: bridge-utils-1.5-9.el7.x86_64 >2014-09-24 10:23:57,273 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: bridge-utils-1.5-9.el7.x86_64. >2014-09-24 10:23:57,273 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: bzip2-1.0.6-12.el7.x86_64 >2014-09-24 10:23:57,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: bzip2-1.0.6-12.el7.x86_64. >2014-09-24 10:23:57,291 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: celt051-0.5.1.3-8.el7.x86_64 >2014-09-24 10:23:57,303 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: celt051-0.5.1.3-8.el7.x86_64. >2014-09-24 10:23:57,303 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: checkpolicy-2.1.12-6.el7.x86_64 >2014-09-24 10:23:57,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: checkpolicy-2.1.12-6.el7.x86_64. >2014-09-24 10:23:57,349 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: cryptsetup-1.6.3-2.el7.x86_64 >2014-09-24 10:23:57,360 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: cryptsetup-1.6.3-2.el7.x86_64. >2014-09-24 10:23:57,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:cups-libs-1.6.3-14.el7.x86_64 >2014-09-24 10:23:57,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:cups-libs-1.6.3-14.el7.x86_64. >2014-09-24 10:23:57,373 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: cyrus-sasl-2.1.26-17.el7.x86_64 >2014-09-24 10:23:57,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: cyrus-sasl-2.1.26-17.el7.x86_64. >2014-09-24 10:23:57,395 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: cyrus-sasl-md5-2.1.26-17.el7.x86_64 >2014-09-24 10:23:57,405 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: cyrus-sasl-md5-2.1.26-17.el7.x86_64. >2014-09-24 10:23:57,448 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: device-mapper-multipath-0.4.9-66.el7.x86_64 >2014-09-24 10:23:57,460 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: device-mapper-multipath-0.4.9-66.el7.x86_64. >2014-09-24 10:23:57,485 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: device-mapper-multipath-libs-0.4.9-66.el7.x86_64 >2014-09-24 10:23:57,496 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: device-mapper-multipath-libs-0.4.9-66.el7.x86_64. >2014-09-24 10:23:57,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: dosfstools-3.0.20-9.el7.x86_64 >2014-09-24 10:23:57,524 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: dosfstools-3.0.20-9.el7.x86_64. >2014-09-24 10:23:57,546 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-all-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,558 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-all-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,579 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-apc-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-apc-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,599 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-apc-snmp-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-apc-snmp-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,629 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-bladecenter-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-bladecenter-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,649 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-brocade-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,659 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-brocade-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,675 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-cisco-mds-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,687 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-cisco-mds-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,696 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-cisco-ucs-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,708 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-cisco-ucs-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,734 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-common-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-common-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,751 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-drac5-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,761 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-drac5-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,767 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-eaton-snmp-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,776 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-eaton-snmp-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,787 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-eps-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-eps-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-hpblade-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-hpblade-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,831 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ibmblade-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,840 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ibmblade-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ifmib-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,874 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ifmib-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ilo-mp-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,915 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ilo-mp-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,947 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ilo2-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,960 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ilo2-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,966 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-intelmodular-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,977 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-intelmodular-4.0.2-21.el7.x86_64. >2014-09-24 10:23:57,986 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ipdu-4.0.2-21.el7.x86_64 >2014-09-24 10:23:57,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ipdu-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,015 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-ipmilan-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,026 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-ipmilan-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,049 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-kdump-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,061 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-kdump-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,088 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-rhevm-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-rhevm-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-rsb-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,114 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-rsb-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,117 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-scsi-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,129 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-scsi-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,147 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-vmware-soap-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,159 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-vmware-soap-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-agents-wti-4.0.2-21.el7.x86_64 >2014-09-24 10:23:58,175 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-agents-wti-4.0.2-21.el7.x86_64. >2014-09-24 10:23:58,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fence-virt-0.3.0-16.el7.x86_64 >2014-09-24 10:23:58,213 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fence-virt-0.3.0-16.el7.x86_64. >2014-09-24 10:23:58,231 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: flac-libs-1.3.0-4.el7.x86_64 >2014-09-24 10:23:58,240 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: flac-libs-1.3.0-4.el7.x86_64. >2014-09-24 10:23:58,243 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fontconfig-2.10.95-7.el7.x86_64 >2014-09-24 10:23:58,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fontconfig-2.10.95-7.el7.x86_64. >2014-09-24 10:23:58,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fontpackages-filesystem-1.44-8.el7.noarch >2014-09-24 10:23:58,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fontpackages-filesystem-1.44-8.el7.noarch. >2014-09-24 10:23:58,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fuse-2.9.2-5.el7.x86_64 >2014-09-24 10:23:58,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fuse-2.9.2-5.el7.x86_64. >2014-09-24 10:23:58,327 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: fuse-libs-2.9.2-5.el7.x86_64 >2014-09-24 10:23:58,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: fuse-libs-2.9.2-5.el7.x86_64. >2014-09-24 10:23:58,362 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gdisk-0.8.6-4.el7.x86_64 >2014-09-24 10:23:58,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gdisk-0.8.6-4.el7.x86_64. >2014-09-24 10:23:58,399 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: genisoimage-1.1.11-22.el7.x86_64 >2014-09-24 10:23:58,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: genisoimage-1.1.11-22.el7.x86_64. >2014-09-24 10:23:58,433 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gfs2-utils-3.1.6-13.el7.x86_64 >2014-09-24 10:23:58,445 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gfs2-utils-3.1.6-13.el7.x86_64. >2014-09-24 10:23:58,511 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ghostscript-9.07-16.el7.x86_64 >2014-09-24 10:23:58,522 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ghostscript-9.07-16.el7.x86_64. >2014-09-24 10:23:58,569 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ghostscript-fonts-5.50-32.el7.noarch >2014-09-24 10:23:58,579 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ghostscript-fonts-5.50-32.el7.noarch. >2014-09-24 10:24:00,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-3.5.2-1.el7.x86_64 >2014-09-24 10:24:00,256 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: glusterfs-3.5.2-1.el7.x86_64. >2014-09-24 10:24:00,999 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-api-3.5.2-1.el7.x86_64 >2014-09-24 10:24:01,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: glusterfs-api-3.5.2-1.el7.x86_64. >2014-09-24 10:24:02,166 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: glusterfs-libs-3.5.2-1.el7.x86_64 >2014-09-24 10:24:02,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: glusterfs-libs-3.5.2-1.el7.x86_64. >2014-09-24 10:24:04,784 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gnutls-3.1.18-9.el7_0.x86_64 >2014-09-24 10:24:04,799 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gnutls-3.1.18-9.el7_0.x86_64. >2014-09-24 10:24:05,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gnutls-dane-3.1.18-9.el7_0.x86_64 >2014-09-24 10:24:05,612 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gnutls-dane-3.1.18-9.el7_0.x86_64. >2014-09-24 10:24:06,598 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gnutls-utils-3.1.18-9.el7_0.x86_64 >2014-09-24 10:24:06,611 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gnutls-utils-3.1.18-9.el7_0.x86_64. >2014-09-24 10:24:06,626 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: gsm-1.0.13-11.el7.x86_64 >2014-09-24 10:24:06,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: gsm-1.0.13-11.el7.x86_64. >2014-09-24 10:24:06,656 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: hexedit-1.2.13-5.el7.x86_64 >2014-09-24 10:24:06,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: hexedit-1.2.13-5.el7.x86_64. >2014-09-24 10:24:06,680 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: hivex-1.3.8-4.el7.x86_64 >2014-09-24 10:24:06,690 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: hivex-1.3.8-4.el7.x86_64. >2014-09-24 10:24:06,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: icoutils-0.31.0-3.el7.x86_64 >2014-09-24 10:24:06,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: icoutils-0.31.0-3.el7.x86_64. >2014-09-24 10:24:06,718 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ioprocess-0.12.0-2.el7.x86_64 >2014-09-24 10:24:06,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ioprocess-0.12.0-2.el7.x86_64. >2014-09-24 10:24:06,758 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ipmitool-1.8.13-7.el7.x86_64 >2014-09-24 10:24:06,769 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ipmitool-1.8.13-7.el7.x86_64. >2014-09-24 10:24:06,781 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: iptables-services-1.4.21-13.el7.x86_64 >2014-09-24 10:24:06,792 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: iptables-services-1.4.21-13.el7.x86_64. >2014-09-24 10:24:06,838 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch >2014-09-24 10:24:06,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch. >2014-09-24 10:24:06,849 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: jasper-libs-1.900.1-26.el7.x86_64 >2014-09-24 10:24:06,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: jasper-libs-1.900.1-26.el7.x86_64. >2014-09-24 10:24:06,857 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: jbigkit-libs-2.0-11.el7.x86_64 >2014-09-24 10:24:06,867 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: jbigkit-libs-2.0-11.el7.x86_64. >2014-09-24 10:24:06,868 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: keyutils-1.5.8-3.el7.x86_64 >2014-09-24 10:24:06,879 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: keyutils-1.5.8-3.el7.x86_64. >2014-09-24 10:24:06,879 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lcms2-2.5-4.el7.x86_64 >2014-09-24 10:24:06,891 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: lcms2-2.5-4.el7.x86_64. >2014-09-24 10:24:06,934 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: ldns-1.6.16-7.el7.x86_64 >2014-09-24 10:24:06,950 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: ldns-1.6.16-7.el7.x86_64. >2014-09-24 10:24:06,950 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libICE-1.0.8-7.el7.x86_64 >2014-09-24 10:24:06,959 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libICE-1.0.8-7.el7.x86_64. >2014-09-24 10:24:06,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libSM-1.2.1-7.el7.x86_64 >2014-09-24 10:24:06,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libSM-1.2.1-7.el7.x86_64. >2014-09-24 10:24:06,971 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libX11-1.6.0-2.1.el7.x86_64 >2014-09-24 10:24:06,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libX11-1.6.0-2.1.el7.x86_64. >2014-09-24 10:24:06,983 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libX11-common-1.6.0-2.1.el7.noarch >2014-09-24 10:24:06,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libX11-common-1.6.0-2.1.el7.noarch. >2014-09-24 10:24:06,997 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXau-1.0.8-2.1.el7.x86_64 >2014-09-24 10:24:07,010 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXau-1.0.8-2.1.el7.x86_64. >2014-09-24 10:24:07,010 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXext-1.3.2-2.1.el7.x86_64 >2014-09-24 10:24:07,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXext-1.3.2-2.1.el7.x86_64. >2014-09-24 10:24:07,023 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXfont-1.4.7-1.1.el7.x86_64 >2014-09-24 10:24:07,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXfont-1.4.7-1.1.el7.x86_64. >2014-09-24 10:24:07,035 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXi-1.7.2-2.1.el7.x86_64 >2014-09-24 10:24:07,047 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXi-1.7.2-2.1.el7.x86_64. >2014-09-24 10:24:07,047 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXt-1.1.4-6.1.el7.x86_64 >2014-09-24 10:24:07,056 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXt-1.1.4-6.1.el7.x86_64. >2014-09-24 10:24:07,056 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libXtst-1.2.2-2.1.el7.x86_64 >2014-09-24 10:24:07,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libXtst-1.2.2-2.1.el7.x86_64. >2014-09-24 10:24:07,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libaio-0.3.109-12.el7.x86_64 >2014-09-24 10:24:07,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libaio-0.3.109-12.el7.x86_64. >2014-09-24 10:24:07,080 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libasyncns-0.8-7.el7.x86_64 >2014-09-24 10:24:07,090 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libasyncns-0.8-7.el7.x86_64. >2014-09-24 10:24:07,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libcgroup-0.41-6.el7.x86_64 >2014-09-24 10:24:07,100 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libcgroup-0.41-6.el7.x86_64. >2014-09-24 10:24:07,111 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libconfig-1.4.9-5.el7.x86_64 >2014-09-24 10:24:07,124 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libconfig-1.4.9-5.el7.x86_64. >2014-09-24 10:24:07,125 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libevent-2.0.21-4.el7.x86_64 >2014-09-24 10:24:07,134 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libevent-2.0.21-4.el7.x86_64. >2014-09-24 10:24:07,135 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libfontenc-1.1.1-5.el7.x86_64 >2014-09-24 10:24:07,143 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libfontenc-1.1.1-5.el7.x86_64. >2014-09-24 10:24:07,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libguestfs-1.22.6-22.el7.x86_64 >2014-09-24 10:24:07,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:libguestfs-1.22.6-22.el7.x86_64. >2014-09-24 10:24:07,313 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libguestfs-tools-c-1.22.6-22.el7.x86_64 >2014-09-24 10:24:07,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:libguestfs-tools-c-1.22.6-22.el7.x86_64. >2014-09-24 10:24:07,352 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libibverbs-1.1.7-6.el7.x86_64 >2014-09-24 10:24:07,364 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libibverbs-1.1.7-6.el7.x86_64. >2014-09-24 10:24:07,551 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libicu-50.1.2-11.el7.x86_64 >2014-09-24 10:24:07,566 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libicu-50.1.2-11.el7.x86_64. >2014-09-24 10:24:07,646 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libiscsi-1.9.0-6.el7.x86_64 >2014-09-24 10:24:07,655 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libiscsi-1.9.0-6.el7.x86_64. >2014-09-24 10:24:07,655 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libjpeg-turbo-1.2.90-5.el7.x86_64 >2014-09-24 10:24:07,666 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libjpeg-turbo-1.2.90-5.el7.x86_64. >2014-09-24 10:24:07,666 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libnfsidmap-0.25-9.el7.x86_64 >2014-09-24 10:24:07,677 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libnfsidmap-0.25-9.el7.x86_64. >2014-09-24 10:24:07,683 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:libogg-1.3.0-7.el7.x86_64 >2014-09-24 10:24:07,692 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 2:libogg-1.3.0-7.el7.x86_64. >2014-09-24 10:24:07,708 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libosinfo-0.2.7-10.el7.x86_64 >2014-09-24 10:24:07,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libosinfo-0.2.7-10.el7.x86_64. >2014-09-24 10:24:07,720 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:libpng-1.5.13-5.el7.x86_64 >2014-09-24 10:24:07,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 2:libpng-1.5.13-5.el7.x86_64. >2014-09-24 10:24:07,748 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: librdmacm-1.0.17.1-1.el7.x86_64 >2014-09-24 10:24:07,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: librdmacm-1.0.17.1-1.el7.x86_64. >2014-09-24 10:24:07,766 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libreport-filesystem-2.1.11-10.el7.x86_64 >2014-09-24 10:24:07,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libreport-filesystem-2.1.11-10.el7.x86_64. >2014-09-24 10:24:07,802 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libseccomp-2.1.1-2.el7.x86_64 >2014-09-24 10:24:07,814 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libseccomp-2.1.1-2.el7.x86_64. >2014-09-24 10:24:07,815 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libsemanage-python-2.1.10-16.el7.x86_64 >2014-09-24 10:24:07,825 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libsemanage-python-2.1.10-16.el7.x86_64. >2014-09-24 10:24:07,843 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libsndfile-1.0.25-9.el7.x86_64 >2014-09-24 10:24:07,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libsndfile-1.0.25-9.el7.x86_64. >2014-09-24 10:24:07,855 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libtiff-4.0.3-14.el7.x86_64 >2014-09-24 10:24:07,865 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libtiff-4.0.3-14.el7.x86_64. >2014-09-24 10:24:07,865 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libtirpc-0.2.4-0.3.el7.x86_64 >2014-09-24 10:24:07,875 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libtirpc-0.2.4-0.3.el7.x86_64. >2014-09-24 10:24:07,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libusal-1.1.11-22.el7.x86_64 >2014-09-24 10:24:07,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libusal-1.1.11-22.el7.x86_64. >2014-09-24 10:24:07,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libusbx-1.0.15-4.el7.x86_64 >2014-09-24 10:24:07,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libusbx-1.0.15-4.el7.x86_64. >2014-09-24 10:24:16,683 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-client-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:16,698 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-client-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:21,167 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:21,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:21,803 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-config-nwfilter-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:21,817 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-config-nwfilter-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:22,750 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-interface-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:22,762 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-interface-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:23,531 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-network-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:23,542 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-network-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:24,269 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-nodedev-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:24,280 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-nodedev-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:24,939 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-nwfilter-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:24,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-nwfilter-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:26,553 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-qemu-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:26,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-qemu-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:27,349 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-secret-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:27,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-secret-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:28,308 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-driver-storage-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:28,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-driver-storage-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:28,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-daemon-kvm-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:28,817 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-daemon-kvm-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:29,461 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-lock-sanlock-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:29,470 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-lock-sanlock-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:30,390 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libvirt-python-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:24:30,400 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libvirt-python-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:24:30,577 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:libvorbis-1.3.3-8.el7.x86_64 >2014-09-24 10:24:30,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:libvorbis-1.3.3-8.el7.x86_64. >2014-09-24 10:24:30,591 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: libxcb-1.9-5.el7.x86_64 >2014-09-24 10:24:30,602 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: libxcb-1.9-5.el7.x86_64. >2014-09-24 10:24:30,616 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lsof-4.87-4.el7.x86_64 >2014-09-24 10:24:30,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: lsof-4.87-4.el7.x86_64. >2014-09-24 10:24:30,641 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lsscsi-0.27-3.el7.x86_64 >2014-09-24 10:24:30,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: lsscsi-0.27-3.el7.x86_64. >2014-09-24 10:24:30,657 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: lzop-1.03-10.el7.x86_64 >2014-09-24 10:24:30,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: lzop-1.03-10.el7.x86_64. >2014-09-24 10:24:30,670 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: mailcap-2.1.41-2.el7.noarch >2014-09-24 10:24:30,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: mailcap-2.1.41-2.el7.noarch. >2014-09-24 10:24:30,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: mdadm-3.2.6-31.el7.x86_64 >2014-09-24 10:24:30,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: mdadm-3.2.6-31.el7.x86_64. >2014-09-24 10:24:30,723 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: mom-0.4.1-1.el7_0.noarch >2014-09-24 10:24:30,732 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: mom-0.4.1-1.el7_0.noarch. >2014-09-24 10:24:30,755 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: mtools-4.0.18-5.el7.x86_64 >2014-09-24 10:24:30,770 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: mtools-4.0.18-5.el7.x86_64. >2014-09-24 10:24:30,811 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:net-snmp-libs-5.7.2-18.el7.x86_64 >2014-09-24 10:24:30,824 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:net-snmp-libs-5.7.2-18.el7.x86_64. >2014-09-24 10:24:30,858 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:net-snmp-utils-5.7.2-18.el7.x86_64 >2014-09-24 10:24:30,867 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:net-snmp-utils-5.7.2-18.el7.x86_64. >2014-09-24 10:24:30,900 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netcf-libs-0.2.3-8.el7.x86_64 >2014-09-24 10:24:30,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: netcf-libs-0.2.3-8.el7.x86_64. >2014-09-24 10:24:30,933 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netpbm-10.61.02-9.el7.x86_64 >2014-09-24 10:24:30,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: netpbm-10.61.02-9.el7.x86_64. >2014-09-24 10:24:30,999 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: netpbm-progs-10.61.02-9.el7.x86_64 >2014-09-24 10:24:31,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: netpbm-progs-10.61.02-9.el7.x86_64. >2014-09-24 10:24:31,031 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:nfs-utils-1.3.0-0.el7.x86_64 >2014-09-24 10:24:31,042 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:nfs-utils-1.3.0-0.el7.x86_64. >2014-09-24 10:24:31,071 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 2:nmap-ncat-6.40-4.el7.x86_64 >2014-09-24 10:24:31,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 2:nmap-ncat-6.40-4.el7.x86_64. >2014-09-24 10:24:31,096 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: numactl-2.0.9-2.el7.x86_64 >2014-09-24 10:24:31,108 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: numactl-2.0.9-2.el7.x86_64. >2014-09-24 10:24:31,113 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: numad-0.5-13.20140225git.el7.x86_64 >2014-09-24 10:24:31,126 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: numad-0.5-13.20140225git.el7.x86_64. >2014-09-24 10:24:31,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Business-ISBN-2.06-2.el7.noarch >2014-09-24 10:24:31,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Business-ISBN-2.06-2.el7.noarch. >2014-09-24 10:24:31,138 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Business-ISBN-Data-20120719.001-2.el7.noarch >2014-09-24 10:24:31,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Business-ISBN-Data-20120719.001-2.el7.noarch. >2014-09-24 10:24:31,150 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Compress-Raw-Bzip2-2.061-3.el7.x86_64 >2014-09-24 10:24:31,163 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Compress-Raw-Bzip2-2.061-3.el7.x86_64. >2014-09-24 10:24:31,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:perl-Compress-Raw-Zlib-2.061-4.el7.x86_64 >2014-09-24 10:24:31,175 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:perl-Compress-Raw-Zlib-2.061-4.el7.x86_64. >2014-09-24 10:24:31,175 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Data-Dumper-2.145-3.el7.x86_64 >2014-09-24 10:24:31,187 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Data-Dumper-2.145-3.el7.x86_64. >2014-09-24 10:24:31,187 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Digest-1.17-245.el7.noarch >2014-09-24 10:24:31,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Digest-1.17-245.el7.noarch. >2014-09-24 10:24:31,199 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Digest-MD5-2.52-3.el7.x86_64 >2014-09-24 10:24:31,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Digest-MD5-2.52-3.el7.x86_64. >2014-09-24 10:24:31,209 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Encode-Locale-1.03-5.el7.noarch >2014-09-24 10:24:31,218 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Encode-Locale-1.03-5.el7.noarch. >2014-09-24 10:24:31,218 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-File-Listing-6.04-7.el7.noarch >2014-09-24 10:24:31,232 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-File-Listing-6.04-7.el7.noarch. >2014-09-24 10:24:31,232 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTML-Parser-3.71-4.el7.x86_64 >2014-09-24 10:24:31,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTML-Parser-3.71-4.el7.x86_64. >2014-09-24 10:24:31,245 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTML-Tagset-3.20-15.el7.noarch >2014-09-24 10:24:31,258 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTML-Tagset-3.20-15.el7.noarch. >2014-09-24 10:24:31,258 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTTP-Cookies-6.01-5.el7.noarch >2014-09-24 10:24:31,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTTP-Cookies-6.01-5.el7.noarch. >2014-09-24 10:24:31,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTTP-Daemon-6.01-5.el7.noarch >2014-09-24 10:24:31,278 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTTP-Daemon-6.01-5.el7.noarch. >2014-09-24 10:24:31,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTTP-Date-6.02-8.el7.noarch >2014-09-24 10:24:31,289 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTTP-Date-6.02-8.el7.noarch. >2014-09-24 10:24:31,289 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTTP-Message-6.06-6.el7.noarch >2014-09-24 10:24:31,309 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTTP-Message-6.06-6.el7.noarch. >2014-09-24 10:24:31,310 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-HTTP-Negotiate-6.01-5.el7.noarch >2014-09-24 10:24:31,320 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-HTTP-Negotiate-6.01-5.el7.noarch. >2014-09-24 10:24:31,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-IO-Compress-2.061-2.el7.noarch >2014-09-24 10:24:31,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-IO-Compress-2.061-2.el7.noarch. >2014-09-24 10:24:31,330 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-IO-HTML-1.00-2.el7.noarch >2014-09-24 10:24:31,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-IO-HTML-1.00-2.el7.noarch. >2014-09-24 10:24:31,339 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-IO-Socket-IP-0.21-4.el7.noarch >2014-09-24 10:24:31,348 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-IO-Socket-IP-0.21-4.el7.noarch. >2014-09-24 10:24:31,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-IO-Socket-SSL-1.94-3.el7.noarch >2014-09-24 10:24:31,357 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-IO-Socket-SSL-1.94-3.el7.noarch. >2014-09-24 10:24:31,358 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-LWP-MediaTypes-6.02-2.el7.noarch >2014-09-24 10:24:31,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-LWP-MediaTypes-6.02-2.el7.noarch. >2014-09-24 10:24:31,368 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Net-HTTP-6.06-2.el7.noarch >2014-09-24 10:24:31,378 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Net-HTTP-6.06-2.el7.noarch. >2014-09-24 10:24:31,378 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Net-LibIDN-0.12-15.el7.x86_64 >2014-09-24 10:24:31,388 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Net-LibIDN-0.12-15.el7.x86_64. >2014-09-24 10:24:31,388 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-Net-SSLeay-1.55-3.el7.x86_64 >2014-09-24 10:24:31,399 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-Net-SSLeay-1.55-3.el7.x86_64. >2014-09-24 10:24:31,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:perl-TimeDate-2.30-2.el7.noarch >2014-09-24 10:24:31,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:perl-TimeDate-2.30-2.el7.noarch. >2014-09-24 10:24:31,410 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-URI-1.60-9.el7.noarch >2014-09-24 10:24:31,418 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-URI-1.60-9.el7.noarch. >2014-09-24 10:24:31,418 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-WWW-RobotRules-6.02-5.el7.noarch >2014-09-24 10:24:31,427 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-WWW-RobotRules-6.02-5.el7.noarch. >2014-09-24 10:24:31,427 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-hivex-1.3.8-4.el7.x86_64 >2014-09-24 10:24:31,435 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-hivex-1.3.8-4.el7.x86_64. >2014-09-24 10:24:31,435 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: perl-libwww-perl-6.05-2.el7.noarch >2014-09-24 10:24:31,445 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: perl-libwww-perl-6.05-2.el7.noarch. >2014-09-24 10:24:31,445 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pexpect-2.3-11.el7.noarch >2014-09-24 10:24:31,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: pexpect-2.3-11.el7.noarch. >2014-09-24 10:24:31,455 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pixman-0.32.4-3.el7.x86_64 >2014-09-24 10:24:31,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: pixman-0.32.4-3.el7.x86_64. >2014-09-24 10:24:31,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pm-utils-1.4.1-26.el7.x86_64 >2014-09-24 10:24:31,476 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: pm-utils-1.4.1-26.el7.x86_64. >2014-09-24 10:24:31,476 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: policycoreutils-python-2.2.5-11.el7.x86_64 >2014-09-24 10:24:31,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: policycoreutils-python-2.2.5-11.el7.x86_64. >2014-09-24 10:24:31,484 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: poppler-data-0.4.6-3.el7.noarch >2014-09-24 10:24:31,493 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: poppler-data-0.4.6-3.el7.noarch. >2014-09-24 10:24:31,493 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: psmisc-22.20-8.el7.x86_64 >2014-09-24 10:24:31,503 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: psmisc-22.20-8.el7.x86_64. >2014-09-24 10:24:31,536 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: pulseaudio-libs-3.0-22.el7.x86_64 >2014-09-24 10:24:31,549 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: pulseaudio-libs-3.0-22.el7.x86_64. >2014-09-24 10:24:31,572 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:pyparted-3.9-7.el7.x86_64 >2014-09-24 10:24:31,581 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:pyparted-3.9-7.el7.x86_64. >2014-09-24 10:24:31,582 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-IPy-0.75-6.el7.noarch >2014-09-24 10:24:31,590 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-IPy-0.75-6.el7.noarch. >2014-09-24 10:24:31,590 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-chardet-2.0.1-7.el7.noarch >2014-09-24 10:24:31,601 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-chardet-2.0.1-7.el7.noarch. >2014-09-24 10:24:31,931 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-cpopen-1.3-4.el7.x86_64 >2014-09-24 10:24:31,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-cpopen-1.3-4.el7.x86_64. >2014-09-24 10:24:31,962 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-inotify-0.9.4-4.el7.noarch >2014-09-24 10:24:31,975 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-inotify-0.9.4-4.el7.noarch. >2014-09-24 10:24:31,975 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-ioprocess-0.12.0-2.el7.noarch >2014-09-24 10:24:31,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-ioprocess-0.12.0-2.el7.noarch. >2014-09-24 10:24:32,050 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-netaddr-0.7.5-7.el7.noarch >2014-09-24 10:24:32,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-netaddr-0.7.5-7.el7.noarch. >2014-09-24 10:24:32,066 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-pthreading-0.1.3-3.el7ev.noarch >2014-09-24 10:24:32,074 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-pthreading-0.1.3-3.el7ev.noarch. >2014-09-24 10:24:32,092 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-requests-1.1.0-8.el7.noarch >2014-09-24 10:24:32,102 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-requests-1.1.0-8.el7.noarch. >2014-09-24 10:24:32,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-six-1.3.0-4.el7.noarch >2014-09-24 10:24:32,137 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-six-1.3.0-4.el7.noarch. >2014-09-24 10:24:32,174 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-suds-0.4.1-5.el7.noarch >2014-09-24 10:24:32,184 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-suds-0.4.1-5.el7.noarch. >2014-09-24 10:24:32,194 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: python-urllib3-1.5-8.el7.noarch >2014-09-24 10:24:32,205 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: python-urllib3-1.5-8.el7.noarch. >2014-09-24 10:24:32,211 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 10:qemu-img-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:24:32,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 10:qemu-img-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:24:32,229 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 10:qemu-kvm-common-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:24:32,241 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 10:qemu-kvm-common-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:24:32,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 10:qemu-kvm-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:24:32,299 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 10:qemu-kvm-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:24:32,322 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 10:qemu-kvm-tools-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:24:32,337 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 10:qemu-kvm-tools-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:24:32,337 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:quota-4.01-11.el7.x86_64 >2014-09-24 10:24:32,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:quota-4.01-11.el7.x86_64. >2014-09-24 10:24:32,347 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:quota-nls-4.01-11.el7.noarch >2014-09-24 10:24:32,358 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:quota-nls-4.01-11.el7.noarch. >2014-09-24 10:24:32,367 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: radvd-1.9.2-7.el7.x86_64 >2014-09-24 10:24:32,381 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: radvd-1.9.2-7.el7.x86_64. >2014-09-24 10:24:32,382 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: rpcbind-0.2.0-23.el7.x86_64 >2014-09-24 10:24:32,403 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: rpcbind-0.2.0-23.el7.x86_64. >2014-09-24 10:24:32,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: rsync-3.0.9-15.el7.x86_64 >2014-09-24 10:24:32,421 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: rsync-3.0.9-15.el7.x86_64. >2014-09-24 10:24:32,422 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: rsyslog-mmjsonparse-7.4.7-6.el7.x86_64 >2014-09-24 10:24:32,435 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: rsyslog-mmjsonparse-7.4.7-6.el7.x86_64. >2014-09-24 10:24:32,459 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-3.1.0-2.el7.x86_64 >2014-09-24 10:24:32,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sanlock-3.1.0-2.el7.x86_64. >2014-09-24 10:24:32,494 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-lib-3.1.0-2.el7.x86_64 >2014-09-24 10:24:32,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sanlock-lib-3.1.0-2.el7.x86_64. >2014-09-24 10:24:32,520 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sanlock-python-3.1.0-2.el7.x86_64 >2014-09-24 10:24:32,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sanlock-python-3.1.0-2.el7.x86_64. >2014-09-24 10:24:32,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: scrub-2.5.2-5.el7.x86_64 >2014-09-24 10:24:32,573 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: scrub-2.5.2-5.el7.x86_64. >2014-09-24 10:24:33,246 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: seabios-bin-1.7.2.2-12.el7_0.1.x86_64 >2014-09-24 10:24:33,260 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: seabios-bin-1.7.2.2-12.el7_0.1.x86_64. >2014-09-24 10:24:33,939 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: seavgabios-bin-1.7.2.2-12.el7_0.1.x86_64 >2014-09-24 10:24:33,952 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: seavgabios-bin-1.7.2.2-12.el7_0.1.x86_64. >2014-09-24 10:24:33,952 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: setools-libs-3.3.7-46.el7.x86_64 >2014-09-24 10:24:33,963 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: setools-libs-3.3.7-46.el7.x86_64. >2014-09-24 10:24:34,020 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sg3_utils-1.37-5.el7.x86_64 >2014-09-24 10:24:34,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sg3_utils-1.37-5.el7.x86_64. >2014-09-24 10:24:34,062 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sg3_utils-libs-1.37-5.el7.x86_64 >2014-09-24 10:24:34,076 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sg3_utils-libs-1.37-5.el7.x86_64. >2014-09-24 10:24:34,090 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:sgabios-bin-0.20110622svn-4.el7.noarch >2014-09-24 10:24:34,103 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:sgabios-bin-0.20110622svn-4.el7.noarch. >2014-09-24 10:24:34,138 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: sos-3.0-23.el7.noarch >2014-09-24 10:24:34,150 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: sos-3.0-23.el7.noarch. >2014-09-24 10:24:34,203 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: spice-server-0.12.4-5.el7.x86_64 >2014-09-24 10:24:34,216 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: spice-server-0.12.4-5.el7.x86_64. >2014-09-24 10:24:34,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: strace-4.8-7.el7.x86_64 >2014-09-24 10:24:34,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: strace-4.8-7.el7.x86_64. >2014-09-24 10:24:34,315 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: supermin-helper-4.1.4-2.el7.x86_64 >2014-09-24 10:24:34,327 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: supermin-helper-4.1.4-2.el7.x86_64. >2014-09-24 10:24:34,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: syslinux-4.05-8.el7.x86_64 >2014-09-24 10:24:34,415 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: syslinux-4.05-8.el7.x86_64. >2014-09-24 10:24:34,471 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: syslinux-extlinux-4.05-8.el7.x86_64 >2014-09-24 10:24:34,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: syslinux-extlinux-4.05-8.el7.x86_64. >2014-09-24 10:24:34,483 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: tcp_wrappers-7.6-77.el7.x86_64 >2014-09-24 10:24:34,493 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: tcp_wrappers-7.6-77.el7.x86_64. >2014-09-24 10:24:34,534 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:telnet-0.17-59.el7.x86_64 >2014-09-24 10:24:34,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:telnet-0.17-59.el7.x86_64. >2014-09-24 10:24:34,566 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: tree-1.6.0-10.el7.x86_64 >2014-09-24 10:24:34,576 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: tree-1.6.0-10.el7.x86_64. >2014-09-24 10:24:35,276 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: tuned-2.3.0-11.el7_0.3.noarch >2014-09-24 10:24:35,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: tuned-2.3.0-11.el7_0.3.noarch. >2014-09-24 10:24:35,341 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: unbound-libs-1.4.20-19.el7.x86_64 >2014-09-24 10:24:35,349 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: unbound-libs-1.4.20-19.el7.x86_64. >2014-09-24 10:24:35,402 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: urw-fonts-2.4-16.el7.noarch >2014-09-24 10:24:35,411 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: urw-fonts-2.4-16.el7.noarch. >2014-09-24 10:24:35,466 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: usbredir-0.6-7.el7.x86_64 >2014-09-24 10:24:35,474 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: usbredir-0.6-7.el7.x86_64. >2014-09-24 10:24:35,485 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-4.16.3-3.el7.beta.x86_64 >2014-09-24 10:24:35,498 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-4.16.3-3.el7.beta.x86_64. >2014-09-24 10:24:35,500 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-cli-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,510 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-cli-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-jsonrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-jsonrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,535 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-python-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,543 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-python-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,543 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,552 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-python-zombiereaper-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,552 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-xmlrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,560 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-xmlrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,560 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: vdsm-yajsonrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:35,568 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: vdsm-yajsonrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:35,568 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: 1:xorg-x11-font-utils-7.5-18.1.el7.x86_64 >2014-09-24 10:24:35,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: 1:xorg-x11-font-utils-7.5-18.1.el7.x86_64. >2014-09-24 10:24:35,605 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Download/Verify: yajl-2.0.4-4.el7.x86_64 >2014-09-24 10:24:35,614 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Download/Verify: yajl-2.0.4-4.el7.x86_64. >2014-09-24 10:24:35,614 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Status: Check Package Signatures >2014-09-24 10:24:35,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Status: Check Package Signatures. >2014-09-24 10:24:35,628 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Status: Running Test Transaction >2014-09-24 10:24:35,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Status: Running Test Transaction. >2014-09-24 10:24:37,896 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Status: Running Transaction >2014-09-24 10:24:37,906 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Status: Running Transaction. >2014-09-24 10:24:39,218 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 1/229: 1:telnet-0.17-59.el7.x86_64 >2014-09-24 10:24:39,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 1/229: 1:telnet-0.17-59.el7.x86_64. >2014-09-24 10:24:39,648 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum update: 2/229: gnutls-3.1.18-9.el7_0.x86_64 >2014-09-24 10:24:39,657 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum update: 2/229: gnutls-3.1.18-9.el7_0.x86_64. >2014-09-24 10:24:41,274 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 3/229: libaio-0.3.109-12.el7.x86_64 >2014-09-24 10:24:41,283 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 3/229: libaio-0.3.109-12.el7.x86_64. >2014-09-24 10:24:41,640 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 4/229: yajl-2.0.4-4.el7.x86_64 >2014-09-24 10:24:41,650 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 4/229: yajl-2.0.4-4.el7.x86_64. >2014-09-24 10:24:41,999 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 5/229: libjpeg-turbo-1.2.90-5.el7.x86_64 >2014-09-24 10:24:42,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 5/229: libjpeg-turbo-1.2.90-5.el7.x86_64. >2014-09-24 10:24:42,383 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 6/229: libICE-1.0.8-7.el7.x86_64 >2014-09-24 10:24:42,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 6/229: libICE-1.0.8-7.el7.x86_64. >2014-09-24 10:24:42,741 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 7/229: 2:libogg-1.3.0-7.el7.x86_64 >2014-09-24 10:24:42,749 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 7/229: 2:libogg-1.3.0-7.el7.x86_64. >2014-09-24 10:24:43,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 8/229: 2:libpng-1.5.13-5.el7.x86_64 >2014-09-24 10:24:43,118 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 8/229: 2:libpng-1.5.13-5.el7.x86_64. >2014-09-24 10:24:43,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 9/229: libSM-1.2.1-7.el7.x86_64 >2014-09-24 10:24:43,535 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 9/229: libSM-1.2.1-7.el7.x86_64. >2014-09-24 10:24:43,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 10/229: sanlock-lib-3.1.0-2.el7.x86_64 >2014-09-24 10:24:43,885 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 10/229: sanlock-lib-3.1.0-2.el7.x86_64. >2014-09-24 10:24:44,201 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 11/229: python-cpopen-1.3-4.el7.x86_64 >2014-09-24 10:24:44,210 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 11/229: python-cpopen-1.3-4.el7.x86_64. >2014-09-24 10:24:44,526 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 12/229: libiscsi-1.9.0-6.el7.x86_64 >2014-09-24 10:24:44,534 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 12/229: libiscsi-1.9.0-6.el7.x86_64. >2014-09-24 10:24:44,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 13/229: 1:perl-Compress-Raw-Zlib-2.061-4.el7.x86_64 >2014-09-24 10:24:44,886 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 13/229: 1:perl-Compress-Raw-Zlib-2.061-4.el7.x86_64. >2014-09-24 10:24:45,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 14/229: fuse-libs-2.9.2-5.el7.x86_64 >2014-09-24 10:24:45,277 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 14/229: fuse-libs-2.9.2-5.el7.x86_64. >2014-09-24 10:24:45,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 15/229: augeas-libs-1.1.0-12.el7.x86_64 >2014-09-24 10:24:45,644 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 15/229: augeas-libs-1.1.0-12.el7.x86_64. >2014-09-24 10:24:46,689 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 16/229: bzip2-1.0.6-12.el7.x86_64 >2014-09-24 10:24:46,700 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 16/229: bzip2-1.0.6-12.el7.x86_64. >2014-09-24 10:24:47,533 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 17/229: sanlock-3.1.0-2.el7.x86_64 >2014-09-24 10:24:47,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 17/229: sanlock-3.1.0-2.el7.x86_64. >2014-09-24 10:24:48,025 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 18/229: perl-hivex-1.3.8-4.el7.x86_64 >2014-09-24 10:24:48,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 18/229: perl-hivex-1.3.8-4.el7.x86_64. >2014-09-24 10:24:48,331 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 19/229: hivex-1.3.8-4.el7.x86_64 >2014-09-24 10:24:48,343 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 19/229: hivex-1.3.8-4.el7.x86_64. >2014-09-24 10:24:49,008 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 20/229: libcgroup-0.41-6.el7.x86_64 >2014-09-24 10:24:49,020 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 20/229: libcgroup-0.41-6.el7.x86_64. >2014-09-24 10:24:49,408 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 21/229: perl-Encode-Locale-1.03-5.el7.noarch >2014-09-24 10:24:49,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 21/229: perl-Encode-Locale-1.03-5.el7.noarch. >2014-09-24 10:24:49,742 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 22/229: perl-Data-Dumper-2.145-3.el7.x86_64 >2014-09-24 10:24:49,751 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 22/229: perl-Data-Dumper-2.145-3.el7.x86_64. >2014-09-24 10:24:50,125 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 23/229: libibverbs-1.1.7-6.el7.x86_64 >2014-09-24 10:24:50,139 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 23/229: libibverbs-1.1.7-6.el7.x86_64. >2014-09-24 10:24:50,542 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 24/229: libevent-2.0.21-4.el7.x86_64 >2014-09-24 10:24:50,553 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 24/229: libevent-2.0.21-4.el7.x86_64. >2014-09-24 10:24:50,976 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 25/229: vdsm-python-zombiereaper-4.16.3-3.el7.beta.noarch >2014-09-24 10:24:50,994 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 25/229: vdsm-python-zombiereaper-4.16.3-3.el7.beta.noarch. >2014-09-24 10:24:51,210 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 26/229: libtirpc-0.2.4-0.3.el7.x86_64 >2014-09-24 10:24:51,219 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 26/229: libtirpc-0.2.4-0.3.el7.x86_64. >2014-09-24 10:24:51,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 27/229: lzop-1.03-10.el7.x86_64 >2014-09-24 10:24:51,571 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 27/229: lzop-1.03-10.el7.x86_64. >2014-09-24 10:24:51,902 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 28/229: libusal-1.1.11-22.el7.x86_64 >2014-09-24 10:24:51,911 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 28/229: libusal-1.1.11-22.el7.x86_64. >2014-09-24 10:24:52,286 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 29/229: libfontenc-1.1.1-5.el7.x86_64 >2014-09-24 10:24:52,295 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 29/229: libfontenc-1.1.1-5.el7.x86_64. >2014-09-24 10:24:52,644 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 30/229: bridge-utils-1.5-9.el7.x86_64 >2014-09-24 10:24:52,654 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 30/229: bridge-utils-1.5-9.el7.x86_64. >2014-09-24 10:24:52,969 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 31/229: dosfstools-3.0.20-9.el7.x86_64 >2014-09-24 10:24:52,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 31/229: dosfstools-3.0.20-9.el7.x86_64. >2014-09-24 10:24:53,354 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 32/229: iptables-services-1.4.21-13.el7.x86_64 >2014-09-24 10:24:53,364 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 32/229: iptables-services-1.4.21-13.el7.x86_64. >2014-09-24 10:24:53,763 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 33/229: libicu-50.1.2-11.el7.x86_64 >2014-09-24 10:24:53,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 33/229: libicu-50.1.2-11.el7.x86_64. >2014-09-24 10:24:55,330 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 34/229: psmisc-22.20-8.el7.x86_64 >2014-09-24 10:24:55,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 34/229: psmisc-22.20-8.el7.x86_64. >2014-09-24 10:24:55,838 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 35/229: pixman-0.32.4-3.el7.x86_64 >2014-09-24 10:24:55,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 35/229: pixman-0.32.4-3.el7.x86_64. >2014-09-24 10:24:56,198 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 36/229: libusbx-1.0.15-4.el7.x86_64 >2014-09-24 10:24:56,209 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 36/229: libusbx-1.0.15-4.el7.x86_64. >2014-09-24 10:24:56,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 37/229: usbredir-0.6-7.el7.x86_64 >2014-09-24 10:24:56,566 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 37/229: usbredir-0.6-7.el7.x86_64. >2014-09-24 10:24:56,924 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 38/229: gdisk-0.8.6-4.el7.x86_64 >2014-09-24 10:24:56,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 38/229: gdisk-0.8.6-4.el7.x86_64. >2014-09-24 10:24:57,358 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 39/229: netcf-libs-0.2.3-8.el7.x86_64 >2014-09-24 10:24:57,368 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 39/229: netcf-libs-0.2.3-8.el7.x86_64. >2014-09-24 10:24:57,875 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 40/229: libXfont-1.4.7-1.1.el7.x86_64 >2014-09-24 10:24:57,885 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 40/229: libXfont-1.4.7-1.1.el7.x86_64. >2014-09-24 10:24:58,251 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 41/229: 1:xorg-x11-font-utils-7.5-18.1.el7.x86_64 >2014-09-24 10:24:58,262 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 41/229: 1:xorg-x11-font-utils-7.5-18.1.el7.x86_64. >2014-09-24 10:24:58,718 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 42/229: genisoimage-1.1.11-22.el7.x86_64 >2014-09-24 10:24:58,731 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 42/229: genisoimage-1.1.11-22.el7.x86_64. >2014-09-24 10:24:59,490 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 43/229: rpcbind-0.2.0-23.el7.x86_64 >2014-09-24 10:24:59,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 43/229: rpcbind-0.2.0-23.el7.x86_64. >2014-09-24 10:24:59,970 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 44/229: librdmacm-1.0.17.1-1.el7.x86_64 >2014-09-24 10:24:59,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 44/229: librdmacm-1.0.17.1-1.el7.x86_64. >2014-09-24 10:25:00,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 45/229: sos-3.0-23.el7.noarch >2014-09-24 10:25:00,344 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 45/229: sos-3.0-23.el7.noarch. >2014-09-24 10:25:01,500 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 46/229: augeas-1.1.0-12.el7.x86_64 >2014-09-24 10:25:01,514 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 46/229: augeas-1.1.0-12.el7.x86_64. >2014-09-24 10:25:01,809 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 47/229: sanlock-python-3.1.0-2.el7.x86_64 >2014-09-24 10:25:01,822 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 47/229: sanlock-python-3.1.0-2.el7.x86_64. >2014-09-24 10:25:02,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 48/229: flac-libs-1.3.0-4.el7.x86_64 >2014-09-24 10:25:02,110 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 48/229: flac-libs-1.3.0-4.el7.x86_64. >2014-09-24 10:25:02,493 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 49/229: 1:libvorbis-1.3.3-8.el7.x86_64 >2014-09-24 10:25:02,502 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 49/229: 1:libvorbis-1.3.3-8.el7.x86_64. >2014-09-24 10:25:02,943 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 50/229: celt051-0.5.1.3-8.el7.x86_64 >2014-09-24 10:25:02,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 50/229: celt051-0.5.1.3-8.el7.x86_64. >2014-09-24 10:25:03,294 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 51/229: spice-server-0.12.4-5.el7.x86_64 >2014-09-24 10:25:03,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 51/229: spice-server-0.12.4-5.el7.x86_64. >2014-09-24 10:25:03,753 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 52/229: jasper-libs-1.900.1-26.el7.x86_64 >2014-09-24 10:25:03,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 52/229: jasper-libs-1.900.1-26.el7.x86_64. >2014-09-24 10:25:04,128 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 53/229: ioprocess-0.12.0-2.el7.x86_64 >2014-09-24 10:25:04,136 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 53/229: ioprocess-0.12.0-2.el7.x86_64. >2014-09-24 10:25:04,436 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 54/229: python-ioprocess-0.12.0-2.el7.noarch >2014-09-24 10:25:04,443 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 54/229: python-ioprocess-0.12.0-2.el7.noarch. >2014-09-24 10:25:04,703 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 55/229: vdsm-python-4.16.3-3.el7.beta.noarch >2014-09-24 10:25:04,711 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 55/229: vdsm-python-4.16.3-3.el7.beta.noarch. >2014-09-24 10:25:05,179 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 56/229: vdsm-xmlrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:25:05,189 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 56/229: vdsm-xmlrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:25:05,797 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 57/229: vdsm-cli-4.16.3-3.el7.beta.noarch >2014-09-24 10:25:05,806 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 57/229: vdsm-cli-4.16.3-3.el7.beta.noarch. >2014-09-24 10:25:06,164 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 58/229: device-mapper-multipath-libs-0.4.9-66.el7.x86_64 >2014-09-24 10:25:06,173 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 58/229: device-mapper-multipath-libs-0.4.9-66.el7.x86_64. >2014-09-24 10:25:06,598 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 59/229: device-mapper-multipath-0.4.9-66.el7.x86_64 >2014-09-24 10:25:06,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 59/229: device-mapper-multipath-0.4.9-66.el7.x86_64. >2014-09-24 10:25:07,056 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 60/229: jbigkit-libs-2.0-11.el7.x86_64 >2014-09-24 10:25:07,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 60/229: jbigkit-libs-2.0-11.el7.x86_64. >2014-09-24 10:25:07,448 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 61/229: libtiff-4.0.3-14.el7.x86_64 >2014-09-24 10:25:07,457 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 61/229: libtiff-4.0.3-14.el7.x86_64. >2014-09-24 10:25:07,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 62/229: 1:pyparted-3.9-7.el7.x86_64 >2014-09-24 10:25:07,849 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 62/229: 1:pyparted-3.9-7.el7.x86_64. >2014-09-24 10:25:08,324 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 63/229: rsyslog-mmjsonparse-7.4.7-6.el7.x86_64 >2014-09-24 10:25:08,333 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 63/229: rsyslog-mmjsonparse-7.4.7-6.el7.x86_64. >2014-09-24 10:25:08,641 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 64/229: glusterfs-libs-3.5.2-1.el7.x86_64 >2014-09-24 10:25:08,649 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 64/229: glusterfs-libs-3.5.2-1.el7.x86_64. >2014-09-24 10:25:09,060 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 65/229: glusterfs-3.5.2-1.el7.x86_64 >2014-09-24 10:25:09,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 65/229: glusterfs-3.5.2-1.el7.x86_64. >2014-09-24 10:25:10,070 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 66/229: glusterfs-api-3.5.2-1.el7.x86_64 >2014-09-24 10:25:10,081 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 66/229: glusterfs-api-3.5.2-1.el7.x86_64. >2014-09-24 10:25:10,678 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 67/229: 10:qemu-img-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:25:10,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 67/229: 10:qemu-img-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:25:11,188 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 68/229: 10:qemu-kvm-common-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:25:11,200 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 68/229: 10:qemu-kvm-common-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:25:12,358 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 69/229: strace-4.8-7.el7.x86_64 >2014-09-24 10:25:12,376 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 69/229: strace-4.8-7.el7.x86_64. >2014-09-24 10:25:12,782 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 70/229: perl-Business-ISBN-Data-20120719.001-2.el7.noarch >2014-09-24 10:25:12,804 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 70/229: perl-Business-ISBN-Data-20120719.001-2.el7.noarch. >2014-09-24 10:25:13,149 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 71/229: perl-Business-ISBN-2.06-2.el7.noarch >2014-09-24 10:25:13,162 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 71/229: perl-Business-ISBN-2.06-2.el7.noarch. >2014-09-24 10:25:13,524 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 72/229: perl-URI-1.60-9.el7.noarch >2014-09-24 10:25:13,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 72/229: perl-URI-1.60-9.el7.noarch. >2014-09-24 10:25:14,033 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 73/229: perl-WWW-RobotRules-6.02-5.el7.noarch >2014-09-24 10:25:14,049 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 73/229: perl-WWW-RobotRules-6.02-5.el7.noarch. >2014-09-24 10:25:14,400 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 74/229: supermin-helper-4.1.4-2.el7.x86_64 >2014-09-24 10:25:14,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 74/229: supermin-helper-4.1.4-2.el7.x86_64. >2014-09-24 10:25:14,876 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 75/229: python-netaddr-0.7.5-7.el7.noarch >2014-09-24 10:25:14,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 75/229: python-netaddr-0.7.5-7.el7.noarch. >2014-09-24 10:25:15,854 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 76/229: python-pthreading-0.1.3-3.el7ev.noarch >2014-09-24 10:25:15,866 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 76/229: python-pthreading-0.1.3-3.el7ev.noarch. >2014-09-24 10:25:16,170 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 77/229: seavgabios-bin-1.7.2.2-12.el7_0.1.x86_64 >2014-09-24 10:25:16,182 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 77/229: seavgabios-bin-1.7.2.2-12.el7_0.1.x86_64. >2014-09-24 10:25:17,013 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 78/229: sg3_utils-libs-1.37-5.el7.x86_64 >2014-09-24 10:25:17,025 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 78/229: sg3_utils-libs-1.37-5.el7.x86_64. >2014-09-24 10:25:17,347 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 79/229: sg3_utils-1.37-5.el7.x86_64 >2014-09-24 10:25:17,359 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 79/229: sg3_utils-1.37-5.el7.x86_64. >2014-09-24 10:25:17,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 80/229: fence-agents-scsi-4.0.2-21.el7.x86_64 >2014-09-24 10:25:17,920 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 80/229: fence-agents-scsi-4.0.2-21.el7.x86_64. >2014-09-24 10:25:18,181 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 81/229: libasyncns-0.8-7.el7.x86_64 >2014-09-24 10:25:18,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 81/229: libasyncns-0.8-7.el7.x86_64. >2014-09-24 10:25:18,480 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 82/229: python-inotify-0.9.4-4.el7.noarch >2014-09-24 10:25:18,490 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 82/229: python-inotify-0.9.4-4.el7.noarch. >2014-09-24 10:25:18,764 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 83/229: pexpect-2.3-11.el7.noarch >2014-09-24 10:25:18,774 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 83/229: pexpect-2.3-11.el7.noarch. >2014-09-24 10:25:19,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 84/229: fence-agents-common-4.0.2-21.el7.x86_64 >2014-09-24 10:25:19,200 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 84/229: fence-agents-common-4.0.2-21.el7.x86_64. >2014-09-24 10:25:19,490 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 85/229: fence-agents-brocade-4.0.2-21.el7.x86_64 >2014-09-24 10:25:19,500 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 85/229: fence-agents-brocade-4.0.2-21.el7.x86_64. >2014-09-24 10:25:19,757 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 86/229: fence-agents-drac5-4.0.2-21.el7.x86_64 >2014-09-24 10:25:19,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 86/229: fence-agents-drac5-4.0.2-21.el7.x86_64. >2014-09-24 10:25:20,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 87/229: fence-agents-apc-4.0.2-21.el7.x86_64 >2014-09-24 10:25:20,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 87/229: fence-agents-apc-4.0.2-21.el7.x86_64. >2014-09-24 10:25:20,324 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 88/229: fence-agents-rhevm-4.0.2-21.el7.x86_64 >2014-09-24 10:25:20,335 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 88/229: fence-agents-rhevm-4.0.2-21.el7.x86_64. >2014-09-24 10:25:20,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 89/229: fence-agents-hpblade-4.0.2-21.el7.x86_64 >2014-09-24 10:25:20,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 89/229: fence-agents-hpblade-4.0.2-21.el7.x86_64. >2014-09-24 10:25:21,042 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 90/229: fence-agents-eps-4.0.2-21.el7.x86_64 >2014-09-24 10:25:21,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 90/229: fence-agents-eps-4.0.2-21.el7.x86_64. >2014-09-24 10:25:21,342 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 91/229: fence-agents-rsb-4.0.2-21.el7.x86_64 >2014-09-24 10:25:21,351 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 91/229: fence-agents-rsb-4.0.2-21.el7.x86_64. >2014-09-24 10:25:21,601 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 92/229: fence-agents-ilo-mp-4.0.2-21.el7.x86_64 >2014-09-24 10:25:21,611 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 92/229: fence-agents-ilo-mp-4.0.2-21.el7.x86_64. >2014-09-24 10:25:21,901 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 93/229: fence-agents-wti-4.0.2-21.el7.x86_64 >2014-09-24 10:25:21,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 93/229: fence-agents-wti-4.0.2-21.el7.x86_64. >2014-09-24 10:25:22,193 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 94/229: fence-agents-cisco-ucs-4.0.2-21.el7.x86_64 >2014-09-24 10:25:22,204 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 94/229: fence-agents-cisco-ucs-4.0.2-21.el7.x86_64. >2014-09-24 10:25:22,452 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 95/229: fence-agents-kdump-4.0.2-21.el7.x86_64 >2014-09-24 10:25:22,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 95/229: fence-agents-kdump-4.0.2-21.el7.x86_64. >2014-09-24 10:25:22,752 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 96/229: fence-agents-bladecenter-4.0.2-21.el7.x86_64 >2014-09-24 10:25:22,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 96/229: fence-agents-bladecenter-4.0.2-21.el7.x86_64. >2014-09-24 10:25:23,043 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 97/229: lcms2-2.5-4.el7.x86_64 >2014-09-24 10:25:23,054 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 97/229: lcms2-2.5-4.el7.x86_64. >2014-09-24 10:25:23,411 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 98/229: tcp_wrappers-7.6-77.el7.x86_64 >2014-09-24 10:25:23,423 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 98/229: tcp_wrappers-7.6-77.el7.x86_64. >2014-09-24 10:25:23,794 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 99/229: 1:perl-TimeDate-2.30-2.el7.noarch >2014-09-24 10:25:23,803 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 99/229: 1:perl-TimeDate-2.30-2.el7.noarch. >2014-09-24 10:25:24,227 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 100/229: perl-HTTP-Date-6.02-8.el7.noarch >2014-09-24 10:25:24,239 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 100/229: perl-HTTP-Date-6.02-8.el7.noarch. >2014-09-24 10:25:24,580 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 101/229: perl-File-Listing-6.04-7.el7.noarch >2014-09-24 10:25:24,593 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 101/229: perl-File-Listing-6.04-7.el7.noarch. >2014-09-24 10:25:24,956 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 102/229: netpbm-10.61.02-9.el7.x86_64 >2014-09-24 10:25:24,967 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 102/229: netpbm-10.61.02-9.el7.x86_64. >2014-09-24 10:25:25,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 103/229: python-suds-0.4.1-5.el7.noarch >2014-09-24 10:25:25,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 103/229: python-suds-0.4.1-5.el7.noarch. >2014-09-24 10:25:26,333 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 104/229: perl-IO-HTML-1.00-2.el7.noarch >2014-09-24 10:25:26,344 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 104/229: perl-IO-HTML-1.00-2.el7.noarch. >2014-09-24 10:25:26,699 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 105/229: mtools-4.0.18-5.el7.x86_64 >2014-09-24 10:25:26,709 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 105/229: mtools-4.0.18-5.el7.x86_64. >2014-09-24 10:25:27,217 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 106/229: syslinux-4.05-8.el7.x86_64 >2014-09-24 10:25:27,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 106/229: syslinux-4.05-8.el7.x86_64. >2014-09-24 10:25:27,918 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 107/229: syslinux-extlinux-4.05-8.el7.x86_64 >2014-09-24 10:25:27,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 107/229: syslinux-extlinux-4.05-8.el7.x86_64. >2014-09-24 10:25:28,413 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 108/229: poppler-data-0.4.6-3.el7.noarch >2014-09-24 10:25:28,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 108/229: poppler-data-0.4.6-3.el7.noarch. >2014-09-24 10:25:29,805 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 109/229: 1:net-snmp-libs-5.7.2-18.el7.x86_64 >2014-09-24 10:25:29,815 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 109/229: 1:net-snmp-libs-5.7.2-18.el7.x86_64. >2014-09-24 10:25:30,467 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 110/229: 1:net-snmp-utils-5.7.2-18.el7.x86_64 >2014-09-24 10:25:30,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 110/229: 1:net-snmp-utils-5.7.2-18.el7.x86_64. >2014-09-24 10:25:31,058 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 111/229: fence-agents-eaton-snmp-4.0.2-21.el7.x86_64 >2014-09-24 10:25:31,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 111/229: fence-agents-eaton-snmp-4.0.2-21.el7.x86_64. >2014-09-24 10:25:31,384 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 112/229: fence-agents-intelmodular-4.0.2-21.el7.x86_64 >2014-09-24 10:25:31,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 112/229: fence-agents-intelmodular-4.0.2-21.el7.x86_64. >2014-09-24 10:25:31,717 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 113/229: fence-agents-ifmib-4.0.2-21.el7.x86_64 >2014-09-24 10:25:31,728 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 113/229: fence-agents-ifmib-4.0.2-21.el7.x86_64. >2014-09-24 10:25:32,001 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 114/229: fence-agents-ibmblade-4.0.2-21.el7.x86_64 >2014-09-24 10:25:32,013 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 114/229: fence-agents-ibmblade-4.0.2-21.el7.x86_64. >2014-09-24 10:25:32,309 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 115/229: fence-agents-ipdu-4.0.2-21.el7.x86_64 >2014-09-24 10:25:32,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 115/229: fence-agents-ipdu-4.0.2-21.el7.x86_64. >2014-09-24 10:25:32,593 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 116/229: fence-agents-apc-snmp-4.0.2-21.el7.x86_64 >2014-09-24 10:25:32,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 116/229: fence-agents-apc-snmp-4.0.2-21.el7.x86_64. >2014-09-24 10:25:32,894 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 117/229: fence-agents-cisco-mds-4.0.2-21.el7.x86_64 >2014-09-24 10:25:32,906 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 117/229: fence-agents-cisco-mds-4.0.2-21.el7.x86_64. >2014-09-24 10:25:33,268 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 118/229: 1:sgabios-bin-0.20110622svn-4.el7.noarch >2014-09-24 10:25:33,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 118/229: 1:sgabios-bin-0.20110622svn-4.el7.noarch. >2014-09-24 10:25:33,553 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 119/229: pm-utils-1.4.1-26.el7.x86_64 >2014-09-24 10:25:33,564 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 119/229: pm-utils-1.4.1-26.el7.x86_64. >2014-09-24 10:25:34,045 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 120/229: numad-0.5-13.20140225git.el7.x86_64 >2014-09-24 10:25:34,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 120/229: numad-0.5-13.20140225git.el7.x86_64. >2014-09-24 10:25:34,472 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 121/229: ldns-1.6.16-7.el7.x86_64 >2014-09-24 10:25:34,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 121/229: ldns-1.6.16-7.el7.x86_64. >2014-09-24 10:25:35,905 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 122/229: unbound-libs-1.4.20-19.el7.x86_64 >2014-09-24 10:25:35,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 122/229: unbound-libs-1.4.20-19.el7.x86_64. >2014-09-24 10:25:36,970 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 123/229: gnutls-dane-3.1.18-9.el7_0.x86_64 >2014-09-24 10:25:36,982 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 123/229: gnutls-dane-3.1.18-9.el7_0.x86_64. >2014-09-24 10:25:37,328 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 124/229: gnutls-utils-3.1.18-9.el7_0.x86_64 >2014-09-24 10:25:37,340 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 124/229: gnutls-utils-3.1.18-9.el7_0.x86_64. >2014-09-24 10:25:37,795 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 125/229: fence-agents-ilo2-4.0.2-21.el7.x86_64 >2014-09-24 10:25:37,807 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 125/229: fence-agents-ilo2-4.0.2-21.el7.x86_64. >2014-09-24 10:25:38,080 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 126/229: libX11-common-1.6.0-2.1.el7.noarch >2014-09-24 10:25:38,091 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 126/229: libX11-common-1.6.0-2.1.el7.noarch. >2014-09-24 10:25:38,647 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 127/229: libosinfo-0.2.7-10.el7.x86_64 >2014-09-24 10:25:38,660 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 127/229: libosinfo-0.2.7-10.el7.x86_64. >2014-09-24 10:25:39,215 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 128/229: 1:quota-nls-4.01-11.el7.noarch >2014-09-24 10:25:39,227 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 128/229: 1:quota-nls-4.01-11.el7.noarch. >2014-09-24 10:25:39,558 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 129/229: 1:quota-4.01-11.el7.x86_64 >2014-09-24 10:25:39,570 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 129/229: 1:quota-4.01-11.el7.x86_64. >2014-09-24 10:25:40,057 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 130/229: cryptsetup-1.6.3-2.el7.x86_64 >2014-09-24 10:25:40,070 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 130/229: cryptsetup-1.6.3-2.el7.x86_64. >2014-09-24 10:25:40,484 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 131/229: keyutils-1.5.8-3.el7.x86_64 >2014-09-24 10:25:40,497 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 131/229: keyutils-1.5.8-3.el7.x86_64. >2014-09-24 10:25:41,153 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 132/229: libsemanage-python-2.1.10-16.el7.x86_64 >2014-09-24 10:25:41,165 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 132/229: libsemanage-python-2.1.10-16.el7.x86_64. >2014-09-24 10:25:41,494 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 133/229: fence-virt-0.3.0-16.el7.x86_64 >2014-09-24 10:25:41,506 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 133/229: fence-virt-0.3.0-16.el7.x86_64. >2014-09-24 10:25:42,003 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 134/229: perl-Net-LibIDN-0.12-15.el7.x86_64 >2014-09-24 10:25:42,015 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 134/229: perl-Net-LibIDN-0.12-15.el7.x86_64. >2014-09-24 10:25:42,403 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 135/229: lsscsi-0.27-3.el7.x86_64 >2014-09-24 10:25:42,417 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 135/229: lsscsi-0.27-3.el7.x86_64. >2014-09-24 10:25:42,820 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 136/229: perl-HTML-Tagset-3.20-15.el7.noarch >2014-09-24 10:25:42,832 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 136/229: perl-HTML-Tagset-3.20-15.el7.noarch. >2014-09-24 10:25:43,187 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 137/229: ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch >2014-09-24 10:25:43,204 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 137/229: ipxe-roms-qemu-20130517-5.gitc4bce43.el7.noarch. >2014-09-24 10:25:43,580 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 138/229: attr-2.4.46-12.el7.x86_64 >2014-09-24 10:25:43,592 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 138/229: attr-2.4.46-12.el7.x86_64. >2014-09-24 10:25:44,006 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 139/229: python-chardet-2.0.1-7.el7.noarch >2014-09-24 10:25:44,018 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 139/229: python-chardet-2.0.1-7.el7.noarch. >2014-09-24 10:25:44,531 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 140/229: numactl-2.0.9-2.el7.x86_64 >2014-09-24 10:25:44,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 140/229: numactl-2.0.9-2.el7.x86_64. >2014-09-24 10:25:44,982 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 141/229: libnfsidmap-0.25-9.el7.x86_64 >2014-09-24 10:25:44,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 141/229: libnfsidmap-0.25-9.el7.x86_64. >2014-09-24 10:25:46,295 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 142/229: 1:nfs-utils-1.3.0-0.el7.x86_64 >2014-09-24 10:25:46,307 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 142/229: 1:nfs-utils-1.3.0-0.el7.x86_64. >2014-09-24 10:25:47,771 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 143/229: gsm-1.0.13-11.el7.x86_64 >2014-09-24 10:25:47,786 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 143/229: gsm-1.0.13-11.el7.x86_64. >2014-09-24 10:25:48,146 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 144/229: libsndfile-1.0.25-9.el7.x86_64 >2014-09-24 10:25:48,158 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 144/229: libsndfile-1.0.25-9.el7.x86_64. >2014-09-24 10:25:48,562 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 145/229: perl-Digest-1.17-245.el7.noarch >2014-09-24 10:25:48,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 145/229: perl-Digest-1.17-245.el7.noarch. >2014-09-24 10:25:48,904 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 146/229: perl-Digest-MD5-2.52-3.el7.x86_64 >2014-09-24 10:25:48,916 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 146/229: perl-Digest-MD5-2.52-3.el7.x86_64. >2014-09-24 10:25:49,272 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 147/229: OpenIPMI-modalias-2.0.19-11.el7.x86_64 >2014-09-24 10:25:49,283 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 147/229: OpenIPMI-modalias-2.0.19-11.el7.x86_64. >2014-09-24 10:25:49,580 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 148/229: ipmitool-1.8.13-7.el7.x86_64 >2014-09-24 10:25:49,591 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 148/229: ipmitool-1.8.13-7.el7.x86_64. >2014-09-24 10:25:50,206 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 149/229: fence-agents-ipmilan-4.0.2-21.el7.x86_64 >2014-09-24 10:25:50,223 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 149/229: fence-agents-ipmilan-4.0.2-21.el7.x86_64. >2014-09-24 10:25:50,564 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 150/229: tree-1.6.0-10.el7.x86_64 >2014-09-24 10:25:50,577 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 150/229: tree-1.6.0-10.el7.x86_64. >2014-09-24 10:25:51,082 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 151/229: 2:nmap-ncat-6.40-4.el7.x86_64 >2014-09-24 10:25:51,097 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 151/229: 2:nmap-ncat-6.40-4.el7.x86_64. >2014-09-24 10:25:51,540 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 152/229: perl-Compress-Raw-Bzip2-2.061-3.el7.x86_64 >2014-09-24 10:25:51,553 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 152/229: perl-Compress-Raw-Bzip2-2.061-3.el7.x86_64. >2014-09-24 10:25:51,941 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 153/229: perl-IO-Compress-2.061-2.el7.noarch >2014-09-24 10:25:51,953 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 153/229: perl-IO-Compress-2.061-2.el7.noarch. >2014-09-24 10:25:52,508 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 154/229: mailcap-2.1.41-2.el7.noarch >2014-09-24 10:25:52,520 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 154/229: mailcap-2.1.41-2.el7.noarch. >2014-09-24 10:25:52,858 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 155/229: perl-LWP-MediaTypes-6.02-2.el7.noarch >2014-09-24 10:25:52,870 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 155/229: perl-LWP-MediaTypes-6.02-2.el7.noarch. >2014-09-24 10:25:53,242 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 156/229: perl-HTTP-Message-6.06-6.el7.noarch >2014-09-24 10:25:53,259 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 156/229: perl-HTTP-Message-6.06-6.el7.noarch. >2014-09-24 10:25:53,626 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 157/229: perl-HTTP-Negotiate-6.01-5.el7.noarch >2014-09-24 10:25:53,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 157/229: perl-HTTP-Negotiate-6.01-5.el7.noarch. >2014-09-24 10:25:53,959 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 158/229: perl-HTTP-Daemon-6.01-5.el7.noarch >2014-09-24 10:25:53,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 158/229: perl-HTTP-Daemon-6.01-5.el7.noarch. >2014-09-24 10:25:54,351 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 159/229: perl-HTTP-Cookies-6.01-5.el7.noarch >2014-09-24 10:25:54,363 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 159/229: perl-HTTP-Cookies-6.01-5.el7.noarch. >2014-09-24 10:25:54,727 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 160/229: perl-HTML-Parser-3.71-4.el7.x86_64 >2014-09-24 10:25:54,737 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 160/229: perl-HTML-Parser-3.71-4.el7.x86_64. >2014-09-24 10:25:55,236 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 161/229: rsync-3.0.9-15.el7.x86_64 >2014-09-24 10:25:55,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 161/229: rsync-3.0.9-15.el7.x86_64. >2014-09-24 10:25:55,931 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 162/229: perl-Net-SSLeay-1.55-3.el7.x86_64 >2014-09-24 10:25:55,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 162/229: perl-Net-SSLeay-1.55-3.el7.x86_64. >2014-09-24 10:25:56,522 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 163/229: fontpackages-filesystem-1.44-8.el7.noarch >2014-09-24 10:25:56,534 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 163/229: fontpackages-filesystem-1.44-8.el7.noarch. >2014-09-24 10:25:56,831 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 164/229: fontconfig-2.10.95-7.el7.x86_64 >2014-09-24 10:25:56,840 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 164/229: fontconfig-2.10.95-7.el7.x86_64. >2014-09-24 10:25:58,467 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 165/229: ghostscript-fonts-5.50-32.el7.noarch >2014-09-24 10:25:58,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 165/229: ghostscript-fonts-5.50-32.el7.noarch. >2014-09-24 10:26:00,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 166/229: urw-fonts-2.4-16.el7.noarch >2014-09-24 10:26:00,087 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 166/229: urw-fonts-2.4-16.el7.noarch. >2014-09-24 10:26:02,830 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 167/229: checkpolicy-2.1.12-6.el7.x86_64 >2014-09-24 10:26:02,841 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 167/229: checkpolicy-2.1.12-6.el7.x86_64. >2014-09-24 10:26:03,252 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 168/229: vdsm-yajsonrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:26:03,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 168/229: vdsm-yajsonrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:26:03,544 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 169/229: vdsm-jsonrpc-4.16.3-3.el7.beta.noarch >2014-09-24 10:26:03,556 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 169/229: vdsm-jsonrpc-4.16.3-3.el7.beta.noarch. >2014-09-24 10:26:03,936 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 170/229: audit-libs-python-2.3.3-4.el7.x86_64 >2014-09-24 10:26:03,949 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 170/229: audit-libs-python-2.3.3-4.el7.x86_64. >2014-09-24 10:26:04,271 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 171/229: 1:cups-libs-1.6.3-14.el7.x86_64 >2014-09-24 10:26:04,283 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 171/229: 1:cups-libs-1.6.3-14.el7.x86_64. >2014-09-24 10:26:04,738 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 172/229: seabios-bin-1.7.2.2-12.el7_0.1.x86_64 >2014-09-24 10:26:04,752 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 172/229: seabios-bin-1.7.2.2-12.el7_0.1.x86_64. >2014-09-24 10:26:05,037 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 173/229: hexedit-1.2.13-5.el7.x86_64 >2014-09-24 10:26:05,048 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 173/229: hexedit-1.2.13-5.el7.x86_64. >2014-09-24 10:26:05,413 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 174/229: scrub-2.5.2-5.el7.x86_64 >2014-09-24 10:26:05,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 174/229: scrub-2.5.2-5.el7.x86_64. >2014-09-24 10:26:05,798 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 175/229: python-IPy-0.75-6.el7.noarch >2014-09-24 10:26:05,809 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 175/229: python-IPy-0.75-6.el7.noarch. >2014-09-24 10:26:06,341 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 176/229: libXau-1.0.8-2.1.el7.x86_64 >2014-09-24 10:26:06,352 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 176/229: libXau-1.0.8-2.1.el7.x86_64. >2014-09-24 10:26:06,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 177/229: libxcb-1.9-5.el7.x86_64 >2014-09-24 10:26:06,745 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 177/229: libxcb-1.9-5.el7.x86_64. >2014-09-24 10:26:07,277 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 178/229: libX11-1.6.0-2.1.el7.x86_64 >2014-09-24 10:26:07,288 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 178/229: libX11-1.6.0-2.1.el7.x86_64. >2014-09-24 10:26:07,777 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 179/229: libXext-1.3.2-2.1.el7.x86_64 >2014-09-24 10:26:07,788 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 179/229: libXext-1.3.2-2.1.el7.x86_64. >2014-09-24 10:26:08,177 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 180/229: libXi-1.7.2-2.1.el7.x86_64 >2014-09-24 10:26:08,187 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 180/229: libXi-1.7.2-2.1.el7.x86_64. >2014-09-24 10:26:08,561 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 181/229: libXtst-1.2.2-2.1.el7.x86_64 >2014-09-24 10:26:08,574 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 181/229: libXtst-1.2.2-2.1.el7.x86_64. >2014-09-24 10:26:08,970 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 182/229: pulseaudio-libs-3.0-22.el7.x86_64 >2014-09-24 10:26:08,983 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 182/229: pulseaudio-libs-3.0-22.el7.x86_64. >2014-09-24 10:26:09,621 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 183/229: libXt-1.1.4-6.1.el7.x86_64 >2014-09-24 10:26:09,633 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 183/229: libXt-1.1.4-6.1.el7.x86_64. >2014-09-24 10:26:10,064 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 184/229: ghostscript-9.07-16.el7.x86_64 >2014-09-24 10:26:10,076 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 184/229: ghostscript-9.07-16.el7.x86_64. >2014-09-24 10:26:12,079 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 185/229: netpbm-progs-10.61.02-9.el7.x86_64 >2014-09-24 10:26:12,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 185/229: netpbm-progs-10.61.02-9.el7.x86_64. >2014-09-24 10:26:13,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 186/229: cyrus-sasl-2.1.26-17.el7.x86_64 >2014-09-24 10:26:13,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 186/229: cyrus-sasl-2.1.26-17.el7.x86_64. >2014-09-24 10:26:14,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 187/229: lsof-4.87-4.el7.x86_64 >2014-09-24 10:26:14,280 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 187/229: lsof-4.87-4.el7.x86_64. >2014-09-24 10:26:14,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 188/229: fuse-2.9.2-5.el7.x86_64 >2014-09-24 10:26:14,735 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 188/229: fuse-2.9.2-5.el7.x86_64. >2014-09-24 10:26:15,140 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 189/229: libreport-filesystem-2.1.11-10.el7.x86_64 >2014-09-24 10:26:15,153 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 189/229: libreport-filesystem-2.1.11-10.el7.x86_64. >2014-09-24 10:26:15,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 190/229: mdadm-3.2.6-31.el7.x86_64 >2014-09-24 10:26:15,466 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 190/229: mdadm-3.2.6-31.el7.x86_64. >2014-09-24 10:26:16,186 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 191/229: libseccomp-2.1.1-2.el7.x86_64 >2014-09-24 10:26:16,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 191/229: libseccomp-2.1.1-2.el7.x86_64. >2014-09-24 10:26:17,136 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 192/229: 10:qemu-kvm-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:26:17,176 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 192/229: 10:qemu-kvm-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:26:17,945 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 193/229: libconfig-1.4.9-5.el7.x86_64 >2014-09-24 10:26:17,956 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 193/229: libconfig-1.4.9-5.el7.x86_64. >2014-09-24 10:26:18,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 194/229: gfs2-utils-3.1.6-13.el7.x86_64 >2014-09-24 10:26:18,323 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 194/229: gfs2-utils-3.1.6-13.el7.x86_64. >2014-09-24 10:26:18,719 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 195/229: python-six-1.3.0-4.el7.noarch >2014-09-24 10:26:18,760 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 195/229: python-six-1.3.0-4.el7.noarch. >2014-09-24 10:26:19,012 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 196/229: python-urllib3-1.5-8.el7.noarch >2014-09-24 10:26:19,023 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 196/229: python-urllib3-1.5-8.el7.noarch. >2014-09-24 10:26:19,353 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 197/229: python-requests-1.1.0-8.el7.noarch >2014-09-24 10:26:19,366 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 197/229: python-requests-1.1.0-8.el7.noarch. >2014-09-24 10:26:19,728 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 198/229: fence-agents-vmware-soap-4.0.2-21.el7.x86_64 >2014-09-24 10:26:19,740 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 198/229: fence-agents-vmware-soap-4.0.2-21.el7.x86_64. >2014-09-24 10:26:20,021 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 199/229: fence-agents-all-4.0.2-21.el7.x86_64 >2014-09-24 10:26:20,033 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 199/229: fence-agents-all-4.0.2-21.el7.x86_64. >2014-09-24 10:26:20,296 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 200/229: cyrus-sasl-md5-2.1.26-17.el7.x86_64 >2014-09-24 10:26:20,308 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 200/229: cyrus-sasl-md5-2.1.26-17.el7.x86_64. >2014-09-24 10:26:20,630 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 201/229: libvirt-client-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:20,644 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 201/229: libvirt-client-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:22,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 202/229: libvirt-daemon-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:22,444 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 202/229: libvirt-daemon-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:23,183 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 203/229: libvirt-daemon-driver-nwfilter-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:23,196 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 203/229: libvirt-daemon-driver-nwfilter-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:23,541 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 204/229: libvirt-python-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:23,551 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 204/229: libvirt-python-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:23,999 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 205/229: libvirt-daemon-driver-interface-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:24,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 205/229: libvirt-daemon-driver-interface-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:24,383 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 206/229: libvirt-daemon-config-nwfilter-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:24,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 206/229: libvirt-daemon-config-nwfilter-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:24,708 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 207/229: libvirt-daemon-driver-storage-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:24,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 207/229: libvirt-daemon-driver-storage-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:25,100 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 208/229: libvirt-lock-sanlock-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:25,112 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 208/229: libvirt-lock-sanlock-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:25,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 209/229: libvirt-daemon-driver-nodedev-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:25,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 209/229: libvirt-daemon-driver-nodedev-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:25,884 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 210/229: libvirt-daemon-driver-secret-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:25,896 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 210/229: libvirt-daemon-driver-secret-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:26,420 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 211/229: setools-libs-3.3.7-46.el7.x86_64 >2014-09-24 10:26:26,431 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 211/229: setools-libs-3.3.7-46.el7.x86_64. >2014-09-24 10:26:26,982 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 212/229: policycoreutils-python-2.2.5-11.el7.x86_64 >2014-09-24 10:26:26,993 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 212/229: policycoreutils-python-2.2.5-11.el7.x86_64. >2014-09-24 10:26:28,017 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 213/229: radvd-1.9.2-7.el7.x86_64 >2014-09-24 10:26:28,029 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 213/229: radvd-1.9.2-7.el7.x86_64. >2014-09-24 10:26:28,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 214/229: libvirt-daemon-driver-network-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:28,567 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 214/229: libvirt-daemon-driver-network-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:28,906 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 215/229: libvirt-daemon-driver-qemu-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:28,915 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 215/229: libvirt-daemon-driver-qemu-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:29,348 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 216/229: libvirt-daemon-kvm-1.1.1-29.el7_0.1.x86_64 >2014-09-24 10:26:29,359 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 216/229: libvirt-daemon-kvm-1.1.1-29.el7_0.1.x86_64. >2014-09-24 10:26:29,606 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 217/229: mom-0.4.1-1.el7_0.noarch >2014-09-24 10:26:29,621 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 217/229: mom-0.4.1-1.el7_0.noarch. >2014-09-24 10:26:30,174 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 218/229: perl-IO-Socket-IP-0.21-4.el7.noarch >2014-09-24 10:26:30,186 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 218/229: perl-IO-Socket-IP-0.21-4.el7.noarch. >2014-09-24 10:26:30,566 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 219/229: perl-IO-Socket-SSL-1.94-3.el7.noarch >2014-09-24 10:26:30,578 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 219/229: perl-IO-Socket-SSL-1.94-3.el7.noarch. >2014-09-24 10:26:31,242 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 220/229: perl-Net-HTTP-6.06-2.el7.noarch >2014-09-24 10:26:31,254 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 220/229: perl-Net-HTTP-6.06-2.el7.noarch. >2014-09-24 10:26:31,651 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 221/229: perl-libwww-perl-6.05-2.el7.noarch >2014-09-24 10:26:31,662 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 221/229: perl-libwww-perl-6.05-2.el7.noarch. >2014-09-24 10:26:32,160 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 222/229: icoutils-0.31.0-3.el7.x86_64 >2014-09-24 10:26:32,171 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 222/229: icoutils-0.31.0-3.el7.x86_64. >2014-09-24 10:26:32,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 223/229: 1:libguestfs-1.22.6-22.el7.x86_64 >2014-09-24 10:26:32,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 223/229: 1:libguestfs-1.22.6-22.el7.x86_64. >2014-09-24 10:26:33,314 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 224/229: 1:libguestfs-tools-c-1.22.6-22.el7.x86_64 >2014-09-24 10:26:33,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 224/229: 1:libguestfs-tools-c-1.22.6-22.el7.x86_64. >2014-09-24 10:26:34,649 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 225/229: vdsm-4.16.3-3.el7.beta.x86_64 >2014-09-24 10:26:34,658 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 225/229: vdsm-4.16.3-3.el7.beta.x86_64. >2014-09-24 10:26:56,691 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum update: 226/229: tuned-2.3.0-11.el7_0.3.noarch >2014-09-24 10:26:56,703 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum update: 226/229: tuned-2.3.0-11.el7_0.3.noarch. >2014-09-24 10:26:57,375 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum install: 227/229: 10:qemu-kvm-tools-rhev-1.5.3-60.el7_0.7.x86_64 >2014-09-24 10:26:57,385 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum install: 227/229: 10:qemu-kvm-tools-rhev-1.5.3-60.el7_0.7.x86_64. >2014-09-24 10:26:57,672 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum updated: 228/229: tuned >2014-09-24 10:26:57,682 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum updated: 228/229: tuned. >2014-09-24 10:26:58,591 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum updated: 229/229: gnutls >2014-09-24 10:26:58,603 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum updated: 229/229: gnutls. >2014-09-24 10:27:01,327 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 1/229: libusbx.x86_64 0:1.0.15-4.el7 - u >2014-09-24 10:27:01,339 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 1/229: libusbx.x86_64 0:1.0.15-4.el7 - u. >2014-09-24 10:27:01,557 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 2/229: hivex.x86_64 0:1.3.8-4.el7 - u >2014-09-24 10:27:01,569 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 2/229: hivex.x86_64 0:1.3.8-4.el7 - u. >2014-09-24 10:27:01,674 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 3/229: sanlock-python.x86_64 0:3.1.0-2.el7 - u >2014-09-24 10:27:01,686 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 3/229: sanlock-python.x86_64 0:3.1.0-2.el7 - u. >2014-09-24 10:27:01,832 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 4/229: perl-WWW-RobotRules.noarch 0:6.02-5.el7 - u >2014-09-24 10:27:01,844 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 4/229: perl-WWW-RobotRules.noarch 0:6.02-5.el7 - u. >2014-09-24 10:27:01,974 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 5/229: libXtst.x86_64 0:1.2.2-2.1.el7 - u >2014-09-24 10:27:01,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 5/229: libXtst.x86_64 0:1.2.2-2.1.el7 - u. >2014-09-24 10:27:02,116 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 6/229: glusterfs-api.x86_64 0:3.5.2-1.el7 - u >2014-09-24 10:27:02,128 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 6/229: glusterfs-api.x86_64 0:3.5.2-1.el7 - u. >2014-09-24 10:27:02,266 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 7/229: perl-IO-Socket-IP.noarch 0:0.21-4.el7 - u >2014-09-24 10:27:02,278 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 7/229: perl-IO-Socket-IP.noarch 0:0.21-4.el7 - u. >2014-09-24 10:27:02,383 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 8/229: fence-agents-brocade.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:02,395 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 8/229: fence-agents-brocade.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:02,516 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 9/229: vdsm-python.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:02,528 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 9/229: vdsm-python.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:02,651 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 10/229: xorg-x11-font-utils.x86_64 1:7.5-18.1.el7 - u >2014-09-24 10:27:02,664 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 10/229: xorg-x11-font-utils.x86_64 1:7.5-18.1.el7 - u. >2014-09-24 10:27:02,786 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 11/229: gnutls.x86_64 0:3.1.18-9.el7_0 - u >2014-09-24 10:27:02,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 11/229: gnutls.x86_64 0:3.1.18-9.el7_0 - u. >2014-09-24 10:27:02,928 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 12/229: pixman.x86_64 0:0.32.4-3.el7 - u >2014-09-24 10:27:02,944 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 12/229: pixman.x86_64 0:0.32.4-3.el7 - u. >2014-09-24 10:27:03,050 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 13/229: libvirt-daemon-driver-interface.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:03,067 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 13/229: libvirt-daemon-driver-interface.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:03,175 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 14/229: perl-HTTP-Date.noarch 0:6.02-8.el7 - u >2014-09-24 10:27:03,190 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 14/229: perl-HTTP-Date.noarch 0:6.02-8.el7 - u. >2014-09-24 10:27:03,292 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 15/229: radvd.x86_64 0:1.9.2-7.el7 - u >2014-09-24 10:27:03,304 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 15/229: radvd.x86_64 0:1.9.2-7.el7 - u. >2014-09-24 10:27:03,417 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 16/229: setools-libs.x86_64 0:3.3.7-46.el7 - u >2014-09-24 10:27:03,429 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 16/229: setools-libs.x86_64 0:3.3.7-46.el7 - u. >2014-09-24 10:27:03,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 17/229: ghostscript.x86_64 0:9.07-16.el7 - u >2014-09-24 10:27:03,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 17/229: ghostscript.x86_64 0:9.07-16.el7 - u. >2014-09-24 10:27:03,676 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 18/229: perl-LWP-MediaTypes.noarch 0:6.02-2.el7 - u >2014-09-24 10:27:03,689 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 18/229: perl-LWP-MediaTypes.noarch 0:6.02-2.el7 - u. >2014-09-24 10:27:03,809 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 19/229: fence-agents-scsi.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:03,820 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 19/229: fence-agents-scsi.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:03,943 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 20/229: bzip2.x86_64 0:1.0.6-12.el7 - u >2014-09-24 10:27:03,954 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 20/229: bzip2.x86_64 0:1.0.6-12.el7 - u. >2014-09-24 10:27:04,068 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 21/229: perl-hivex.x86_64 0:1.3.8-4.el7 - u >2014-09-24 10:27:04,080 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 21/229: perl-hivex.x86_64 0:1.3.8-4.el7 - u. >2014-09-24 10:27:04,176 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 22/229: psmisc.x86_64 0:22.20-8.el7 - u >2014-09-24 10:27:04,188 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 22/229: psmisc.x86_64 0:22.20-8.el7 - u. >2014-09-24 10:27:04,310 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 23/229: perl-IO-Socket-SSL.noarch 0:1.94-3.el7 - u >2014-09-24 10:27:04,325 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 23/229: perl-IO-Socket-SSL.noarch 0:1.94-3.el7 - u. >2014-09-24 10:27:04,443 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 24/229: libvirt-daemon-config-nwfilter.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:04,455 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 24/229: libvirt-daemon-config-nwfilter.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:04,568 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 25/229: pulseaudio-libs.x86_64 0:3.0-22.el7 - u >2014-09-24 10:27:04,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 25/229: pulseaudio-libs.x86_64 0:3.0-22.el7 - u. >2014-09-24 10:27:04,702 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 26/229: cyrus-sasl-md5.x86_64 0:2.1.26-17.el7 - u >2014-09-24 10:27:04,714 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 26/229: cyrus-sasl-md5.x86_64 0:2.1.26-17.el7 - u. >2014-09-24 10:27:04,827 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 27/229: python-requests.noarch 0:1.1.0-8.el7 - u >2014-09-24 10:27:04,837 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 27/229: python-requests.noarch 0:1.1.0-8.el7 - u. >2014-09-24 10:27:04,960 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 28/229: libicu.x86_64 0:50.1.2-11.el7 - u >2014-09-24 10:27:04,971 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 28/229: libicu.x86_64 0:50.1.2-11.el7 - u. >2014-09-24 10:27:05,102 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 29/229: perl-HTTP-Negotiate.noarch 0:6.01-5.el7 - u >2014-09-24 10:27:05,115 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 29/229: perl-HTTP-Negotiate.noarch 0:6.01-5.el7 - u. >2014-09-24 10:27:05,236 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 30/229: fence-agents-eaton-snmp.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:05,249 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 30/229: fence-agents-eaton-snmp.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:05,377 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 31/229: python-six.noarch 0:1.3.0-4.el7 - u >2014-09-24 10:27:05,389 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 31/229: python-six.noarch 0:1.3.0-4.el7 - u. >2014-09-24 10:27:05,504 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 32/229: gfs2-utils.x86_64 0:3.1.6-13.el7 - u >2014-09-24 10:27:05,516 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 32/229: gfs2-utils.x86_64 0:3.1.6-13.el7 - u. >2014-09-24 10:27:05,636 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 33/229: vdsm.x86_64 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:05,648 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 33/229: vdsm.x86_64 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:05,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 34/229: libvirt-daemon-driver-qemu.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:05,782 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 34/229: libvirt-daemon-driver-qemu.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:05,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 35/229: libconfig.x86_64 0:1.4.9-5.el7 - u >2014-09-24 10:27:05,923 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 35/229: libconfig.x86_64 0:1.4.9-5.el7 - u. >2014-09-24 10:27:06,045 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 36/229: gnutls-dane.x86_64 0:3.1.18-9.el7_0 - u >2014-09-24 10:27:06,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 36/229: gnutls-dane.x86_64 0:3.1.18-9.el7_0 - u. >2014-09-24 10:27:06,162 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 37/229: libseccomp.x86_64 0:2.1.1-2.el7 - u >2014-09-24 10:27:06,174 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 37/229: libseccomp.x86_64 0:2.1.1-2.el7 - u. >2014-09-24 10:27:06,698 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 38/229: libreport-filesystem.x86_64 0:2.1.11-10.el7 - u >2014-09-24 10:27:06,710 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 38/229: libreport-filesystem.x86_64 0:2.1.11-10.el7 - u. >2014-09-24 10:27:06,839 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 39/229: libSM.x86_64 0:1.2.1-7.el7 - u >2014-09-24 10:27:06,852 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 39/229: libSM.x86_64 0:1.2.1-7.el7 - u. >2014-09-24 10:27:06,989 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 40/229: yajl.x86_64 0:2.0.4-4.el7 - u >2014-09-24 10:27:07,000 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 40/229: yajl.x86_64 0:2.0.4-4.el7 - u. >2014-09-24 10:27:07,148 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 41/229: augeas-libs.x86_64 0:1.1.0-12.el7 - u >2014-09-24 10:27:07,158 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 41/229: augeas-libs.x86_64 0:1.1.0-12.el7 - u. >2014-09-24 10:27:07,265 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 42/229: fuse.x86_64 0:2.9.2-5.el7 - u >2014-09-24 10:27:07,276 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 42/229: fuse.x86_64 0:2.9.2-5.el7 - u. >2014-09-24 10:27:07,398 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 43/229: lsof.x86_64 0:4.87-4.el7 - u >2014-09-24 10:27:07,411 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 43/229: lsof.x86_64 0:4.87-4.el7 - u. >2014-09-24 10:27:07,525 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 44/229: cyrus-sasl.x86_64 0:2.1.26-17.el7 - u >2014-09-24 10:27:07,541 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 44/229: cyrus-sasl.x86_64 0:2.1.26-17.el7 - u. >2014-09-24 10:27:07,657 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 45/229: vdsm-jsonrpc.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:07,668 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 45/229: vdsm-jsonrpc.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:07,799 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 46/229: syslinux-extlinux.x86_64 0:4.05-8.el7 - u >2014-09-24 10:27:07,816 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 46/229: syslinux-extlinux.x86_64 0:4.05-8.el7 - u. >2014-09-24 10:27:07,932 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 47/229: fence-agents-common.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:07,945 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 47/229: fence-agents-common.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:08,041 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 48/229: perl-Net-HTTP.noarch 0:6.06-2.el7 - u >2014-09-24 10:27:08,053 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 48/229: perl-Net-HTTP.noarch 0:6.06-2.el7 - u. >2014-09-24 10:27:08,166 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 49/229: jasper-libs.x86_64 0:1.900.1-26.el7 - u >2014-09-24 10:27:08,177 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 49/229: jasper-libs.x86_64 0:1.900.1-26.el7 - u. >2014-09-24 10:27:08,299 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 50/229: libXau.x86_64 0:1.0.8-2.1.el7 - u >2014-09-24 10:27:08,311 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 50/229: libXau.x86_64 0:1.0.8-2.1.el7 - u. >2014-09-24 10:27:08,433 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 51/229: python-IPy.noarch 0:0.75-6.el7 - u >2014-09-24 10:27:08,451 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 51/229: python-IPy.noarch 0:0.75-6.el7 - u. >2014-09-24 10:27:08,575 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 52/229: fence-agents-intelmodular.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:08,587 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 52/229: fence-agents-intelmodular.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:08,700 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 53/229: scrub.x86_64 0:2.5.2-5.el7 - u >2014-09-24 10:27:08,714 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 53/229: scrub.x86_64 0:2.5.2-5.el7 - u. >2014-09-24 10:27:08,833 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 54/229: hexedit.x86_64 0:1.2.13-5.el7 - u >2014-09-24 10:27:08,843 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 54/229: hexedit.x86_64 0:1.2.13-5.el7 - u. >2014-09-24 10:27:08,967 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 55/229: seabios-bin.x86_64 0:1.7.2.2-12.el7_0.1 - u >2014-09-24 10:27:08,978 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 55/229: seabios-bin.x86_64 0:1.7.2.2-12.el7_0.1 - u. >2014-09-24 10:27:09,100 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 56/229: libvirt-daemon-kvm.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:09,111 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 56/229: libvirt-daemon-kvm.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:09,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 57/229: fence-agents-drac5.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:09,245 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 57/229: fence-agents-drac5.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:09,367 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 58/229: cups-libs.x86_64 1:1.6.3-14.el7 - u >2014-09-24 10:27:09,379 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 58/229: cups-libs.x86_64 1:1.6.3-14.el7 - u. >2014-09-24 10:27:09,500 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 59/229: fence-agents-apc.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:09,512 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 59/229: fence-agents-apc.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:09,626 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 60/229: usbredir.x86_64 0:0.6-7.el7 - u >2014-09-24 10:27:09,637 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 60/229: usbredir.x86_64 0:0.6-7.el7 - u. >2014-09-24 10:27:09,759 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 61/229: audit-libs-python.x86_64 0:2.3.3-4.el7 - u >2014-09-24 10:27:09,771 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 61/229: audit-libs-python.x86_64 0:2.3.3-4.el7 - u. >2014-09-24 10:27:09,885 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 62/229: sanlock.x86_64 0:3.1.0-2.el7 - u >2014-09-24 10:27:09,896 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 62/229: sanlock.x86_64 0:3.1.0-2.el7 - u. >2014-09-24 10:27:09,993 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 63/229: vdsm-yajsonrpc.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:10,004 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 63/229: vdsm-yajsonrpc.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:10,126 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 64/229: checkpolicy.x86_64 0:2.1.12-6.el7 - u >2014-09-24 10:27:10,138 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 64/229: checkpolicy.x86_64 0:2.1.12-6.el7 - u. >2014-09-24 10:27:10,260 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 65/229: syslinux.x86_64 0:4.05-8.el7 - u >2014-09-24 10:27:10,272 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 65/229: syslinux.x86_64 0:4.05-8.el7 - u. >2014-09-24 10:27:10,393 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 66/229: fontpackages-filesystem.noarch 0:1.44-8.el7 - u >2014-09-24 10:27:10,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 66/229: fontpackages-filesystem.noarch 0:1.44-8.el7 - u. >2014-09-24 10:27:10,527 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 67/229: augeas.x86_64 0:1.1.0-12.el7 - u >2014-09-24 10:27:10,537 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 67/229: augeas.x86_64 0:1.1.0-12.el7 - u. >2014-09-24 10:27:10,660 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 68/229: perl-Net-SSLeay.x86_64 0:1.55-3.el7 - u >2014-09-24 10:27:10,671 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 68/229: perl-Net-SSLeay.x86_64 0:1.55-3.el7 - u. >2014-09-24 10:27:10,802 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 69/229: fence-agents-ifmib.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:10,812 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 69/229: fence-agents-ifmib.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:10,927 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 70/229: rsync.x86_64 0:3.0.9-15.el7 - u >2014-09-24 10:27:10,939 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 70/229: rsync.x86_64 0:3.0.9-15.el7 - u. >2014-09-24 10:27:11,061 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 71/229: fence-agents-rhevm.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:11,073 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 71/229: fence-agents-rhevm.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:11,194 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 72/229: mailcap.noarch 0:2.1.41-2.el7 - u >2014-09-24 10:27:11,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 72/229: mailcap.noarch 0:2.1.41-2.el7 - u. >2014-09-24 10:27:11,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 73/229: libtiff.x86_64 0:4.0.3-14.el7 - u >2014-09-24 10:27:11,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 73/229: libtiff.x86_64 0:4.0.3-14.el7 - u. >2014-09-24 10:27:11,461 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 74/229: fence-agents-vmware-soap.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:11,472 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 74/229: fence-agents-vmware-soap.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:11,603 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 75/229: perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7 - u >2014-09-24 10:27:11,615 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 75/229: perl-Compress-Raw-Bzip2.x86_64 0:2.061-3.el7 - u. >2014-09-24 10:27:11,745 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 76/229: nmap-ncat.x86_64 2:6.40-4.el7 - u >2014-09-24 10:27:11,757 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 76/229: nmap-ncat.x86_64 2:6.40-4.el7 - u. >2014-09-24 10:27:11,862 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 77/229: netpbm-progs.x86_64 0:10.61.02-9.el7 - u >2014-09-24 10:27:11,876 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 77/229: netpbm-progs.x86_64 0:10.61.02-9.el7 - u. >2014-09-24 10:27:11,987 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 78/229: libpng.x86_64 2:1.5.13-5.el7 - u >2014-09-24 10:27:11,997 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 78/229: libpng.x86_64 2:1.5.13-5.el7 - u. >2014-09-24 10:27:12,120 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 79/229: iptables-services.x86_64 0:1.4.21-13.el7 - u >2014-09-24 10:27:12,129 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 79/229: iptables-services.x86_64 0:1.4.21-13.el7 - u. >2014-09-24 10:27:12,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 80/229: tree.x86_64 0:1.6.0-10.el7 - u >2014-09-24 10:27:12,263 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 80/229: tree.x86_64 0:1.6.0-10.el7 - u. >2014-09-24 10:27:12,395 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 81/229: qemu-kvm-rhev.x86_64 10:1.5.3-60.el7_0.7 - u >2014-09-24 10:27:12,405 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 81/229: qemu-kvm-rhev.x86_64 10:1.5.3-60.el7_0.7 - u. >2014-09-24 10:27:12,512 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 82/229: OpenIPMI-modalias.x86_64 0:2.0.19-11.el7 - u >2014-09-24 10:27:12,522 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 82/229: OpenIPMI-modalias.x86_64 0:2.0.19-11.el7 - u. >2014-09-24 10:27:12,629 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 83/229: dosfstools.x86_64 0:3.0.20-9.el7 - u >2014-09-24 10:27:12,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 83/229: dosfstools.x86_64 0:3.0.20-9.el7 - u. >2014-09-24 10:27:12,762 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 84/229: perl-Digest.noarch 0:1.17-245.el7 - u >2014-09-24 10:27:12,772 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 84/229: perl-Digest.noarch 0:1.17-245.el7 - u. >2014-09-24 10:27:12,887 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 85/229: fuse-libs.x86_64 0:2.9.2-5.el7 - u >2014-09-24 10:27:12,896 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 85/229: fuse-libs.x86_64 0:2.9.2-5.el7 - u. >2014-09-24 10:27:13,029 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 86/229: flac-libs.x86_64 0:1.3.0-4.el7 - u >2014-09-24 10:27:13,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 86/229: flac-libs.x86_64 0:1.3.0-4.el7 - u. >2014-09-24 10:27:13,163 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 87/229: fence-agents-all.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:13,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 87/229: fence-agents-all.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:13,280 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 88/229: vdsm-xmlrpc.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:13,290 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 88/229: vdsm-xmlrpc.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:13,413 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 89/229: perl-HTTP-Daemon.noarch 0:6.01-5.el7 - u >2014-09-24 10:27:13,424 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 89/229: perl-HTTP-Daemon.noarch 0:6.01-5.el7 - u. >2014-09-24 10:27:13,530 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 90/229: gsm.x86_64 0:1.0.13-11.el7 - u >2014-09-24 10:27:13,542 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 90/229: gsm.x86_64 0:1.0.13-11.el7 - u. >2014-09-24 10:27:13,680 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 91/229: libnfsidmap.x86_64 0:0.25-9.el7 - u >2014-09-24 10:27:13,691 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 91/229: libnfsidmap.x86_64 0:0.25-9.el7 - u. >2014-09-24 10:27:13,839 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 92/229: numactl.x86_64 0:2.0.9-2.el7 - u >2014-09-24 10:27:13,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 92/229: numactl.x86_64 0:2.0.9-2.el7 - u. >2014-09-24 10:27:13,997 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 93/229: ghostscript-fonts.noarch 0:5.50-32.el7 - u >2014-09-24 10:27:14,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 93/229: ghostscript-fonts.noarch 0:5.50-32.el7 - u. >2014-09-24 10:27:14,143 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 94/229: libvirt-daemon.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:14,152 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 94/229: libvirt-daemon.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:14,257 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 95/229: libogg.x86_64 2:1.3.0-7.el7 - u >2014-09-24 10:27:14,268 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 95/229: libogg.x86_64 2:1.3.0-7.el7 - u. >2014-09-24 10:27:14,399 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 96/229: python-chardet.noarch 0:2.0.1-7.el7 - u >2014-09-24 10:27:14,411 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 96/229: python-chardet.noarch 0:2.0.1-7.el7 - u. >2014-09-24 10:27:14,508 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 97/229: fence-agents-hpblade.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:14,519 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 97/229: fence-agents-hpblade.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:14,658 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 98/229: attr.x86_64 0:2.4.46-12.el7 - u >2014-09-24 10:27:14,667 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 98/229: attr.x86_64 0:2.4.46-12.el7 - u. >2014-09-24 10:27:14,791 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 99/229: ipxe-roms-qemu.noarch 0:20130517-5.gitc4bce43.el7 - u >2014-09-24 10:27:14,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 99/229: ipxe-roms-qemu.noarch 0:20130517-5.gitc4bce43.el7 - u. >2014-09-24 10:27:14,925 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 100/229: bridge-utils.x86_64 0:1.5-9.el7 - u >2014-09-24 10:27:14,937 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 100/229: bridge-utils.x86_64 0:1.5-9.el7 - u. >2014-09-24 10:27:15,075 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 101/229: libvorbis.x86_64 1:1.3.3-8.el7 - u >2014-09-24 10:27:15,087 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 101/229: libvorbis.x86_64 1:1.3.3-8.el7 - u. >2014-09-24 10:27:15,183 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 102/229: perl-HTML-Tagset.noarch 0:3.20-15.el7 - u >2014-09-24 10:27:15,195 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 102/229: perl-HTML-Tagset.noarch 0:3.20-15.el7 - u. >2014-09-24 10:27:15,317 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 103/229: libXt.x86_64 0:1.1.4-6.1.el7 - u >2014-09-24 10:27:15,330 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 103/229: libXt.x86_64 0:1.1.4-6.1.el7 - u. >2014-09-24 10:27:15,467 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 104/229: fence-agents-eps.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:15,478 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 104/229: fence-agents-eps.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:15,609 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 105/229: telnet.x86_64 1:0.17-59.el7 - u >2014-09-24 10:27:15,621 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 105/229: telnet.x86_64 1:0.17-59.el7 - u. >2014-09-24 10:27:15,751 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 106/229: lsscsi.x86_64 0:0.27-3.el7 - u >2014-09-24 10:27:15,763 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 106/229: lsscsi.x86_64 0:0.27-3.el7 - u. >2014-09-24 10:27:15,893 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 107/229: libfontenc.x86_64 0:1.1.1-5.el7 - u >2014-09-24 10:27:15,904 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 107/229: libfontenc.x86_64 0:1.1.1-5.el7 - u. >2014-09-24 10:27:16,026 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 108/229: perl-Net-LibIDN.x86_64 0:0.12-15.el7 - u >2014-09-24 10:27:16,038 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 108/229: perl-Net-LibIDN.x86_64 0:0.12-15.el7 - u. >2014-09-24 10:27:16,168 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 109/229: fence-virt.x86_64 0:0.3.0-16.el7 - u >2014-09-24 10:27:16,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 109/229: fence-virt.x86_64 0:0.3.0-16.el7 - u. >2014-09-24 10:27:16,310 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 110/229: libsemanage-python.x86_64 0:2.1.10-16.el7 - u >2014-09-24 10:27:16,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 110/229: libsemanage-python.x86_64 0:2.1.10-16.el7 - u. >2014-09-24 10:27:16,460 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 111/229: keyutils.x86_64 0:1.5.8-3.el7 - u >2014-09-24 10:27:16,471 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 111/229: keyutils.x86_64 0:1.5.8-3.el7 - u. >2014-09-24 10:27:16,585 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 112/229: libvirt-daemon-driver-storage.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:16,597 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 112/229: libvirt-daemon-driver-storage.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:17,302 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 113/229: perl-HTTP-Cookies.noarch 0:6.01-5.el7 - u >2014-09-24 10:27:17,314 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 113/229: perl-HTTP-Cookies.noarch 0:6.01-5.el7 - u. >2014-09-24 10:27:17,452 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 114/229: python-urllib3.noarch 0:1.5-8.el7 - u >2014-09-24 10:27:17,464 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 114/229: python-urllib3.noarch 0:1.5-8.el7 - u. >2014-09-24 10:27:17,610 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 115/229: libvirt-daemon-driver-network.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:17,622 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 115/229: libvirt-daemon-driver-network.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:17,752 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 116/229: perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7 - u >2014-09-24 10:27:17,764 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 116/229: perl-Compress-Raw-Zlib.x86_64 1:2.061-4.el7 - u. >2014-09-24 10:27:17,877 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 117/229: fence-agents-ilo2.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:17,887 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 117/229: fence-agents-ilo2.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:18,027 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 118/229: cryptsetup.x86_64 0:1.6.3-2.el7 - u >2014-09-24 10:27:18,039 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 118/229: cryptsetup.x86_64 0:1.6.3-2.el7 - u. >2014-09-24 10:27:18,161 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 119/229: fence-agents-rsb.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:18,172 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 119/229: fence-agents-rsb.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:18,311 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 120/229: fence-agents-ilo-mp.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:18,322 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 120/229: fence-agents-ilo-mp.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:18,461 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 121/229: quota-nls.noarch 1:4.01-11.el7 - u >2014-09-24 10:27:18,473 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 121/229: quota-nls.noarch 1:4.01-11.el7 - u. >2014-09-24 10:27:18,594 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 122/229: perl-Business-ISBN.noarch 0:2.06-2.el7 - u >2014-09-24 10:27:18,605 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 122/229: perl-Business-ISBN.noarch 0:2.06-2.el7 - u. >2014-09-24 10:27:18,711 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 123/229: libjpeg-turbo.x86_64 0:1.2.90-5.el7 - u >2014-09-24 10:27:18,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 123/229: libjpeg-turbo.x86_64 0:1.2.90-5.el7 - u. >2014-09-24 10:27:18,845 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 124/229: libosinfo.x86_64 0:0.2.7-10.el7 - u >2014-09-24 10:27:18,857 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 124/229: libosinfo.x86_64 0:0.2.7-10.el7 - u. >2014-09-24 10:27:18,986 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 125/229: libX11-common.noarch 0:1.6.0-2.1.el7 - u >2014-09-24 10:27:19,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 125/229: libX11-common.noarch 0:1.6.0-2.1.el7 - u. >2014-09-24 10:27:19,137 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 126/229: ldns.x86_64 0:1.6.16-7.el7 - u >2014-09-24 10:27:19,148 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 126/229: ldns.x86_64 0:1.6.16-7.el7 - u. >2014-09-24 10:27:19,279 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 127/229: numad.x86_64 0:0.5-13.20140225git.el7 - u >2014-09-24 10:27:19,291 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 127/229: numad.x86_64 0:0.5-13.20140225git.el7 - u. >2014-09-24 10:27:19,395 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 128/229: libusal.x86_64 0:1.1.11-22.el7 - u >2014-09-24 10:27:19,410 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 128/229: libusal.x86_64 0:1.1.11-22.el7 - u. >2014-09-24 10:27:19,520 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 129/229: pm-utils.x86_64 0:1.4.1-26.el7 - u >2014-09-24 10:27:19,533 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 129/229: pm-utils.x86_64 0:1.4.1-26.el7 - u. >2014-09-24 10:27:19,662 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 130/229: qemu-kvm-common-rhev.x86_64 10:1.5.3-60.el7_0.7 - u >2014-09-24 10:27:19,674 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 130/229: qemu-kvm-common-rhev.x86_64 10:1.5.3-60.el7_0.7 - u. >2014-09-24 10:27:19,796 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 131/229: qemu-kvm-tools-rhev.x86_64 10:1.5.3-60.el7_0.7 - u >2014-09-24 10:27:19,808 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 131/229: qemu-kvm-tools-rhev.x86_64 10:1.5.3-60.el7_0.7 - u. >2014-09-24 10:27:19,937 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 132/229: libiscsi.x86_64 0:1.9.0-6.el7 - u >2014-09-24 10:27:19,946 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 132/229: libiscsi.x86_64 0:1.9.0-6.el7 - u. >2014-09-24 10:27:20,063 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 133/229: sgabios-bin.noarch 1:0.20110622svn-4.el7 - u >2014-09-24 10:27:20,072 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 133/229: sgabios-bin.noarch 1:0.20110622svn-4.el7 - u. >2014-09-24 10:27:20,204 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 134/229: lzop.x86_64 0:1.03-10.el7 - u >2014-09-24 10:27:20,217 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 134/229: lzop.x86_64 0:1.03-10.el7 - u. >2014-09-24 10:27:20,329 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 135/229: perl-HTML-Parser.x86_64 0:3.71-4.el7 - u >2014-09-24 10:27:20,342 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 135/229: perl-HTML-Parser.x86_64 0:3.71-4.el7 - u. >2014-09-24 10:27:20,463 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 136/229: fence-agents-wti.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:20,475 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 136/229: fence-agents-wti.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:20,596 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 137/229: libXi.x86_64 0:1.7.2-2.1.el7 - u >2014-09-24 10:27:20,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 137/229: libXi.x86_64 0:1.7.2-2.1.el7 - u. >2014-09-24 10:27:20,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 138/229: perl-Digest-MD5.x86_64 0:2.52-3.el7 - u >2014-09-24 10:27:20,733 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 138/229: perl-Digest-MD5.x86_64 0:2.52-3.el7 - u. >2014-09-24 10:27:20,863 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 139/229: libvirt-client.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:20,875 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 139/229: libvirt-client.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:20,997 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 140/229: net-snmp-libs.x86_64 1:5.7.2-18.el7 - u >2014-09-24 10:27:21,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 140/229: net-snmp-libs.x86_64 1:5.7.2-18.el7 - u. >2014-09-24 10:27:21,130 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 141/229: perl-HTTP-Message.noarch 0:6.06-6.el7 - u >2014-09-24 10:27:21,142 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 141/229: perl-HTTP-Message.noarch 0:6.06-6.el7 - u. >2014-09-24 10:27:21,272 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 142/229: gdisk.x86_64 0:0.8.6-4.el7 - u >2014-09-24 10:27:21,284 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 142/229: gdisk.x86_64 0:0.8.6-4.el7 - u. >2014-09-24 10:27:21,414 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 143/229: fence-agents-ipmilan.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:21,425 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 143/229: fence-agents-ipmilan.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:21,581 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 144/229: net-snmp-utils.x86_64 1:5.7.2-18.el7 - u >2014-09-24 10:27:21,593 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 144/229: net-snmp-utils.x86_64 1:5.7.2-18.el7 - u. >2014-09-24 10:27:21,722 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 145/229: libvirt-python.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:21,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 145/229: libvirt-python.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:21,881 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 146/229: libtirpc.x86_64 0:0.2.4-0.3.el7 - u >2014-09-24 10:27:21,892 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 146/229: libtirpc.x86_64 0:0.2.4-0.3.el7 - u. >2014-09-24 10:27:22,023 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 147/229: libxcb.x86_64 0:1.9-5.el7 - u >2014-09-24 10:27:22,035 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 147/229: libxcb.x86_64 0:1.9-5.el7 - u. >2014-09-24 10:27:22,156 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 148/229: urw-fonts.noarch 0:2.4-16.el7 - u >2014-09-24 10:27:22,169 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 148/229: urw-fonts.noarch 0:2.4-16.el7 - u. >2014-09-24 10:27:22,316 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 149/229: vdsm-python-zombiereaper.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:22,328 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 149/229: vdsm-python-zombiereaper.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:22,473 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 150/229: device-mapper-multipath.x86_64 0:0.4.9-66.el7 - u >2014-09-24 10:27:22,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 150/229: device-mapper-multipath.x86_64 0:0.4.9-66.el7 - u. >2014-09-24 10:27:22,639 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 151/229: libevent.x86_64 0:2.0.21-4.el7 - u >2014-09-24 10:27:22,652 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 151/229: libevent.x86_64 0:2.0.21-4.el7 - u. >2014-09-24 10:27:22,773 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 152/229: mom.noarch 0:0.4.1-1.el7_0 - u >2014-09-24 10:27:22,784 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 152/229: mom.noarch 0:0.4.1-1.el7_0 - u. >2014-09-24 10:27:22,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 153/229: poppler-data.noarch 0:0.4.6-3.el7 - u >2014-09-24 10:27:22,909 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 153/229: poppler-data.noarch 0:0.4.6-3.el7 - u. >2014-09-24 10:27:23,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 154/229: mtools.x86_64 0:4.0.18-5.el7 - u >2014-09-24 10:27:23,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 154/229: mtools.x86_64 0:4.0.18-5.el7 - u. >2014-09-24 10:27:23,190 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 155/229: glusterfs.x86_64 0:3.5.2-1.el7 - u >2014-09-24 10:27:23,202 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 155/229: glusterfs.x86_64 0:3.5.2-1.el7 - u. >2014-09-24 10:27:23,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 156/229: libguestfs-tools-c.x86_64 1:1.22.6-22.el7 - u >2014-09-24 10:27:23,342 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 156/229: libguestfs-tools-c.x86_64 1:1.22.6-22.el7 - u. >2014-09-24 10:27:23,449 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 157/229: perl-IO-HTML.noarch 0:1.00-2.el7 - u >2014-09-24 10:27:23,461 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 157/229: perl-IO-HTML.noarch 0:1.00-2.el7 - u. >2014-09-24 10:27:23,584 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 158/229: python-suds.noarch 0:0.4.1-5.el7 - u >2014-09-24 10:27:23,595 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 158/229: python-suds.noarch 0:0.4.1-5.el7 - u. >2014-09-24 10:27:23,709 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 159/229: netpbm.x86_64 0:10.61.02-9.el7 - u >2014-09-24 10:27:23,720 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 159/229: netpbm.x86_64 0:10.61.02-9.el7 - u. >2014-09-24 10:27:23,842 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 160/229: genisoimage.x86_64 0:1.1.11-22.el7 - u >2014-09-24 10:27:23,855 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 160/229: genisoimage.x86_64 0:1.1.11-22.el7 - u. >2014-09-24 10:27:23,992 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 161/229: perl-TimeDate.noarch 1:2.30-2.el7 - u >2014-09-24 10:27:24,008 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 161/229: perl-TimeDate.noarch 1:2.30-2.el7 - u. >2014-09-24 10:27:24,101 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 162/229: celt051.x86_64 0:0.5.1.3-8.el7 - u >2014-09-24 10:27:24,113 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 162/229: celt051.x86_64 0:0.5.1.3-8.el7 - u. >2014-09-24 10:27:24,226 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 163/229: perl-File-Listing.noarch 0:6.04-7.el7 - u >2014-09-24 10:27:24,237 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 163/229: perl-File-Listing.noarch 0:6.04-7.el7 - u. >2014-09-24 10:27:24,351 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 164/229: fence-agents-ibmblade.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:24,360 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 164/229: fence-agents-ibmblade.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:24,476 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 165/229: tcp_wrappers.x86_64 0:7.6-77.el7 - u >2014-09-24 10:27:24,485 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 165/229: tcp_wrappers.x86_64 0:7.6-77.el7 - u. >2014-09-24 10:27:24,635 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 166/229: fence-agents-ipdu.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:24,643 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 166/229: fence-agents-ipdu.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:24,760 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 167/229: sos.noarch 0:3.0-23.el7 - u >2014-09-24 10:27:24,768 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 167/229: sos.noarch 0:3.0-23.el7 - u. >2014-09-24 10:27:24,885 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 168/229: libvirt-lock-sanlock.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:24,893 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 168/229: libvirt-lock-sanlock.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:25,018 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 169/229: lcms2.x86_64 0:2.5-4.el7 - u >2014-09-24 10:27:25,027 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 169/229: lcms2.x86_64 0:2.5-4.el7 - u. >2014-09-24 10:27:25,152 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 170/229: policycoreutils-python.x86_64 0:2.2.5-11.el7 - u >2014-09-24 10:27:25,166 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 170/229: policycoreutils-python.x86_64 0:2.2.5-11.el7 - u. >2014-09-24 10:27:25,332 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 171/229: nfs-utils.x86_64 1:1.3.0-0.el7 - u >2014-09-24 10:27:25,341 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 171/229: nfs-utils.x86_64 1:1.3.0-0.el7 - u. >2014-09-24 10:27:25,477 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 172/229: libsndfile.x86_64 0:1.0.25-9.el7 - u >2014-09-24 10:27:25,487 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 172/229: libsndfile.x86_64 0:1.0.25-9.el7 - u. >2014-09-24 10:27:25,627 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 173/229: gnutls-utils.x86_64 0:3.1.18-9.el7_0 - u >2014-09-24 10:27:25,636 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 173/229: gnutls-utils.x86_64 0:3.1.18-9.el7_0 - u. >2014-09-24 10:27:25,769 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 174/229: pexpect.noarch 0:2.3-11.el7 - u >2014-09-24 10:27:25,777 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 174/229: pexpect.noarch 0:2.3-11.el7 - u. >2014-09-24 10:27:25,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 175/229: libvirt-daemon-driver-nwfilter.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:25,919 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 175/229: libvirt-daemon-driver-nwfilter.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:26,036 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 176/229: ipmitool.x86_64 0:1.8.13-7.el7 - u >2014-09-24 10:27:26,046 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 176/229: ipmitool.x86_64 0:1.8.13-7.el7 - u. >2014-09-24 10:27:26,161 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 177/229: perl-IO-Compress.noarch 0:2.061-2.el7 - u >2014-09-24 10:27:26,170 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 177/229: perl-IO-Compress.noarch 0:2.061-2.el7 - u. >2014-09-24 10:27:26,303 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 178/229: unbound-libs.x86_64 0:1.4.20-19.el7 - u >2014-09-24 10:27:26,313 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 178/229: unbound-libs.x86_64 0:1.4.20-19.el7 - u. >2014-09-24 10:27:26,662 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 179/229: python-inotify.noarch 0:0.9.4-4.el7 - u >2014-09-24 10:27:26,674 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 179/229: python-inotify.noarch 0:0.9.4-4.el7 - u. >2014-09-24 10:27:26,804 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 180/229: fontconfig.x86_64 0:2.10.95-7.el7 - u >2014-09-24 10:27:26,814 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 180/229: fontconfig.x86_64 0:2.10.95-7.el7 - u. >2014-09-24 10:27:26,938 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 181/229: libaio.x86_64 0:0.3.109-12.el7 - u >2014-09-24 10:27:26,947 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 181/229: libaio.x86_64 0:0.3.109-12.el7 - u. >2014-09-24 10:27:27,054 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 182/229: fence-agents-cisco-ucs.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:27,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 182/229: fence-agents-cisco-ucs.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:27,180 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 183/229: vdsm-cli.noarch 0:4.16.3-3.el7.beta - u >2014-09-24 10:27:27,191 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 183/229: vdsm-cli.noarch 0:4.16.3-3.el7.beta - u. >2014-09-24 10:27:27,321 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 184/229: sg3_utils.x86_64 0:1.37-5.el7 - u >2014-09-24 10:27:27,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 184/229: sg3_utils.x86_64 0:1.37-5.el7 - u. >2014-09-24 10:27:27,455 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 185/229: libvirt-daemon-driver-nodedev.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:27,463 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 185/229: libvirt-daemon-driver-nodedev.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:27,588 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 186/229: rpcbind.x86_64 0:0.2.0-23.el7 - u >2014-09-24 10:27:27,600 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 186/229: rpcbind.x86_64 0:0.2.0-23.el7 - u. >2014-09-24 10:27:27,713 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 187/229: libXfont.x86_64 0:1.4.7-1.1.el7 - u >2014-09-24 10:27:27,725 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 187/229: libXfont.x86_64 0:1.4.7-1.1.el7 - u. >2014-09-24 10:27:27,847 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 188/229: libasyncns.x86_64 0:0.8-7.el7 - u >2014-09-24 10:27:27,860 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 188/229: libasyncns.x86_64 0:0.8-7.el7 - u. >2014-09-24 10:27:27,980 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 189/229: libibverbs.x86_64 0:1.1.7-6.el7 - u >2014-09-24 10:27:27,995 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 189/229: libibverbs.x86_64 0:1.1.7-6.el7 - u. >2014-09-24 10:27:28,139 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 190/229: glusterfs-libs.x86_64 0:3.5.2-1.el7 - u >2014-09-24 10:27:28,154 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 190/229: glusterfs-libs.x86_64 0:3.5.2-1.el7 - u. >2014-09-24 10:27:28,297 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 191/229: device-mapper-multipath-libs.x86_64 0:0.4.9-66.el7 - u >2014-09-24 10:27:28,310 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 191/229: device-mapper-multipath-libs.x86_64 0:0.4.9-66.el7 - u. >2014-09-24 10:27:28,439 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 192/229: perl-libwww-perl.noarch 0:6.05-2.el7 - u >2014-09-24 10:27:28,451 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 192/229: perl-libwww-perl.noarch 0:6.05-2.el7 - u. >2014-09-24 10:27:28,598 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 193/229: sg3_utils-libs.x86_64 0:1.37-5.el7 - u >2014-09-24 10:27:28,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 193/229: sg3_utils-libs.x86_64 0:1.37-5.el7 - u. >2014-09-24 10:27:28,748 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 194/229: seavgabios-bin.x86_64 0:1.7.2.2-12.el7_0.1 - u >2014-09-24 10:27:28,758 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 194/229: seavgabios-bin.x86_64 0:1.7.2.2-12.el7_0.1 - u. >2014-09-24 10:27:28,898 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 195/229: libvirt-daemon-driver-secret.x86_64 0:1.1.1-29.el7_0.1 - u >2014-09-24 10:27:28,908 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 195/229: libvirt-daemon-driver-secret.x86_64 0:1.1.1-29.el7_0.1 - u. >2014-09-24 10:27:29,048 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 196/229: perl-Data-Dumper.x86_64 0:2.145-3.el7 - u >2014-09-24 10:27:29,057 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 196/229: perl-Data-Dumper.x86_64 0:2.145-3.el7 - u. >2014-09-24 10:27:29,181 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 197/229: perl-Encode-Locale.noarch 0:1.03-5.el7 - u >2014-09-24 10:27:29,192 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 197/229: perl-Encode-Locale.noarch 0:1.03-5.el7 - u. >2014-09-24 10:27:29,324 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 198/229: quota.x86_64 1:4.01-11.el7 - u >2014-09-24 10:27:29,332 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 198/229: quota.x86_64 1:4.01-11.el7 - u. >2014-09-24 10:27:29,432 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 199/229: python-pthreading.noarch 0:0.1.3-3.el7ev - u >2014-09-24 10:27:29,446 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 199/229: python-pthreading.noarch 0:0.1.3-3.el7ev - u. >2014-09-24 10:27:29,573 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 200/229: python-netaddr.noarch 0:0.7.5-7.el7 - u >2014-09-24 10:27:29,585 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 200/229: python-netaddr.noarch 0:0.7.5-7.el7 - u. >2014-09-24 10:27:29,707 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 201/229: supermin-helper.x86_64 0:4.1.4-2.el7 - u >2014-09-24 10:27:29,718 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 201/229: supermin-helper.x86_64 0:4.1.4-2.el7 - u. >2014-09-24 10:27:29,841 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 202/229: python-ioprocess.noarch 0:0.12.0-2.el7 - u >2014-09-24 10:27:29,851 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 202/229: python-ioprocess.noarch 0:0.12.0-2.el7 - u. >2014-09-24 10:27:29,949 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 203/229: netcf-libs.x86_64 0:0.2.3-8.el7 - u >2014-09-24 10:27:29,961 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 203/229: netcf-libs.x86_64 0:0.2.3-8.el7 - u. >2014-09-24 10:27:30,057 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 204/229: fence-agents-apc-snmp.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:30,069 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 204/229: fence-agents-apc-snmp.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:30,199 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 205/229: perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7 - u >2014-09-24 10:27:30,211 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 205/229: perl-Business-ISBN-Data.noarch 0:20120719.001-2.el7 - u. >2014-09-24 10:27:30,333 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 206/229: librdmacm.x86_64 0:1.0.17.1-1.el7 - u >2014-09-24 10:27:30,346 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 206/229: librdmacm.x86_64 0:1.0.17.1-1.el7 - u. >2014-09-24 10:27:30,474 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 207/229: libICE.x86_64 0:1.0.8-7.el7 - u >2014-09-24 10:27:30,484 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 207/229: libICE.x86_64 0:1.0.8-7.el7 - u. >2014-09-24 10:27:30,599 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 208/229: tuned.noarch 0:2.3.0-11.el7_0.3 - u >2014-09-24 10:27:30,608 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 208/229: tuned.noarch 0:2.3.0-11.el7_0.3 - u. >2014-09-24 10:27:30,733 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 209/229: python-cpopen.x86_64 0:1.3-4.el7 - u >2014-09-24 10:27:30,743 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 209/229: python-cpopen.x86_64 0:1.3-4.el7 - u. >2014-09-24 10:27:30,850 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 210/229: strace.x86_64 0:4.8-7.el7 - u >2014-09-24 10:27:30,859 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 210/229: strace.x86_64 0:4.8-7.el7 - u. >2014-09-24 10:27:30,975 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 211/229: rsyslog-mmjsonparse.x86_64 0:7.4.7-6.el7 - u >2014-09-24 10:27:30,986 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 211/229: rsyslog-mmjsonparse.x86_64 0:7.4.7-6.el7 - u. >2014-09-24 10:27:31,108 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 212/229: pyparted.x86_64 1:3.9-7.el7 - u >2014-09-24 10:27:31,117 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 212/229: pyparted.x86_64 1:3.9-7.el7 - u. >2014-09-24 10:27:31,234 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 213/229: libguestfs.x86_64 1:1.22.6-22.el7 - u >2014-09-24 10:27:31,244 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 213/229: libguestfs.x86_64 1:1.22.6-22.el7 - u. >2014-09-24 10:27:31,527 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 214/229: libX11.x86_64 0:1.6.0-2.1.el7 - u >2014-09-24 10:27:31,545 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 214/229: libX11.x86_64 0:1.6.0-2.1.el7 - u. >2014-09-24 10:27:31,661 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 215/229: jbigkit-libs.x86_64 0:2.0-11.el7 - u >2014-09-24 10:27:31,670 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 215/229: jbigkit-libs.x86_64 0:2.0-11.el7 - u. >2014-09-24 10:27:31,786 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 216/229: mdadm.x86_64 0:3.2.6-31.el7 - u >2014-09-24 10:27:31,800 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 216/229: mdadm.x86_64 0:3.2.6-31.el7 - u. >2014-09-24 10:27:31,911 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 217/229: spice-server.x86_64 0:0.12.4-5.el7 - u >2014-09-24 10:27:31,921 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 217/229: spice-server.x86_64 0:0.12.4-5.el7 - u. >2014-09-24 10:27:32,044 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 218/229: fence-agents-kdump.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:32,064 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 218/229: fence-agents-kdump.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:32,186 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 219/229: qemu-img-rhev.x86_64 10:1.5.3-60.el7_0.7 - u >2014-09-24 10:27:32,199 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 219/229: qemu-img-rhev.x86_64 10:1.5.3-60.el7_0.7 - u. >2014-09-24 10:27:32,320 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 220/229: fence-agents-bladecenter.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:32,334 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 220/229: fence-agents-bladecenter.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:32,470 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 221/229: libcgroup.x86_64 0:0.41-6.el7 - u >2014-09-24 10:27:32,483 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 221/229: libcgroup.x86_64 0:0.41-6.el7 - u. >2014-09-24 10:27:32,595 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 222/229: libXext.x86_64 0:1.3.2-2.1.el7 - u >2014-09-24 10:27:32,607 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 222/229: libXext.x86_64 0:1.3.2-2.1.el7 - u. >2014-09-24 10:27:32,720 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 223/229: sanlock-lib.x86_64 0:3.1.0-2.el7 - u >2014-09-24 10:27:32,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 223/229: sanlock-lib.x86_64 0:3.1.0-2.el7 - u. >2014-09-24 10:27:32,862 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 224/229: perl-URI.noarch 0:1.60-9.el7 - u >2014-09-24 10:27:32,872 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 224/229: perl-URI.noarch 0:1.60-9.el7 - u. >2014-09-24 10:27:32,987 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 225/229: icoutils.x86_64 0:0.31.0-3.el7 - u >2014-09-24 10:27:32,998 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 225/229: icoutils.x86_64 0:0.31.0-3.el7 - u. >2014-09-24 10:27:33,129 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 226/229: fence-agents-cisco-mds.x86_64 0:4.0.2-21.el7 - u >2014-09-24 10:27:33,141 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 226/229: fence-agents-cisco-mds.x86_64 0:4.0.2-21.el7 - u. >2014-09-24 10:27:33,254 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 227/229: ioprocess.x86_64 0:0.12.0-2.el7 - u >2014-09-24 10:27:33,265 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 227/229: ioprocess.x86_64 0:0.12.0-2.el7 - u. >2014-09-24 10:27:33,380 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 228/229: gnutls.x86_64 0:3.1.18-8.el7 - ud >2014-09-24 10:27:33,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 228/229: gnutls.x86_64 0:3.1.18-8.el7 - ud. >2014-09-24 10:27:33,393 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Yum Verify: 229/229: tuned.noarch 0:2.3.0-11.el7 - ud >2014-09-24 10:27:33,404 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Yum Verify: 229/229: tuned.noarch 0:2.3.0-11.el7 - ud. >2014-09-24 10:27:33,714 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Misc configuration >2014-09-24 10:27:33,727 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Misc configuration. >2014-09-24 10:27:33,812 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Setting time >2014-09-24 10:27:33,827 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Setting time. >2014-09-24 10:27:36,269 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Enrolling certificate >2014-09-24 10:27:36,281 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Enrolling certificate. >2014-09-24 10:27:39,572 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Transaction commit >2014-09-24 10:27:39,582 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Transaction commit. >2014-09-24 10:27:39,725 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Closing up >2014-09-24 10:27:39,734 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Closing up. >2014-09-24 10:27:42,046 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stopping libvirtd >2014-09-24 10:27:42,055 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stopping libvirtd. >2014-09-24 10:27:42,086 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Starting vdsm >2014-09-24 10:27:42,096 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Starting vdsm. >2014-09-24 10:27:46,051 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Pre-termination >2014-09-24 10:27:46,059 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Pre-termination. >2014-09-24 10:27:46,093 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140924102746-slot-7.rhev.lab.eng.brq.redhat.com-1029b7df.log' >2014-09-24 10:27:46,106 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-20140924102746-slot-7.rhev.lab.eng.brq.redhat.com-1029b7df.log'. >2014-09-24 10:27:46,996 INFO [org.ovirt.engine.core.bll.InstallerMessages] (VdsDeploy) Installation slot-7.rhev.lab.eng.brq.redhat.com: Stage: Termination >2014-09-24 10:27:47,009 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) Correlation ID: 1029b7df, Call Stack: null, Custom Event ID: -1, Message: Installing Host blade. Stage: Termination. >2014-09-24 10:27:47,478 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-24 10:27:47,479 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-24 10:27:47,480 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) Retry failed: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-24 10:27:47,480 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (org.ovirt.thread.pool-6-thread-15) Exception during connection: java.util.concurrent.ExecutionException: java.net.ConnectException: Connection refused > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.get(FutureTask.java:188) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient.connect(ReactorClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.getClient(JsonRpcClient.java:101) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.JsonRpcClient.call(JsonRpcClient.java:58) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.FutureMap.<init>(FutureMap.java:60) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:918) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$3.call(JsonRpcVdsServer.java:914) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:405) [vdsbroker.jar:] > at org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer$FutureCallable.call(JsonRpcVdsServer.java:394) [vdsbroker.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [rt.jar:1.7.0_65] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_65] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_65] > at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_65] >Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.Net.connect0(Native Method) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:465) [rt.jar:1.7.0_65] > at sun.nio.ch.Net.connect(Net.java:457) [rt.jar:1.7.0_65] > at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:670) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:94) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient$1.call(ReactorClient.java:84) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable.call(Retryable.java:26) [vdsm-jsonrpc-java-client.jar:] > at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_65] > at org.ovirt.vdsm.jsonrpc.client.utils.ReactorScheduler.performPendingOperations(ReactorScheduler.java:28) [vdsm-jsonrpc-java-client.jar:] > at org.ovirt.vdsm.jsonrpc.client.reactors.Reactor.run(Reactor.java:58) [vdsm-jsonrpc-java-client.jar:] > >2014-09-24 10:27:47,482 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-24 10:27:47,484 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Command PollVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb) execution failed. Exception: RuntimeException: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request >2014-09-24 10:27:47,985 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-24 10:27:48,093 INFO [org.ovirt.engine.core.bll.network.NetworkConfigurator] (org.ovirt.thread.pool-6-thread-20) [1029b7df] Engine managed to communicate with VDSM agent on host blade >2014-09-24 10:27:48,811 INFO [org.ovirt.engine.core.bll.network.host.SetupNetworksCommand] (org.ovirt.thread.pool-6-thread-20) [6c53baba] Running command: SetupNetworksCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-24 10:27:48,814 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-20) [6c53baba] START, SetupNetworksVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, force=false, checkConnectivity=true, conectivityTimeout=120, > networks=[rhevm {id=00000000-0000-0000-0000-000000000009, description=Management Network, comment=null, subnet=null, gateway=null, type=null, vlanId=null, stp=false, dataCenterId=00000002-0002-0002-0002-0000000001ad, mtu=0, vmNetwork=true, cluster=NetworkCluster {id={clusterId=null, networkId=null}, status=OPERATIONAL, display=true, required=true, migration=true}, providedBy=null, label=null, qosId=null}], > bonds=[], > interfaces=[em1 {id=8f16d4e0-d0d5-4469-ab88-6796fadbbf24, vdsId=96620dcc-2509-48fe-841b-85247771c4fb, macAddress=84:8f:69:67:1f:0f, networkName=rhevm, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=10.34.63.137, subnet=255.255.252.0, gateway=null, mtu=1500, bridged=false, speed=1000, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}, > em2 {id=6915be9d-b86e-43ad-9f73-4dfc363f900e, vdsId=96620dcc-2509-48fe-841b-85247771c4fb, macAddress=84:8f:69:67:1f:11, networkName=null, baseInterface=null, vlanId=null, bonded=null, bondName=null, bondOptions=null, bootProtocol=DHCP, address=, subnet=, gateway=null, mtu=1500, bridged=false, speed=0, type=0, networkImplementationDetails=null, qos=null qosOverridden=false, labels=null, customProperties=null}], > removedNetworks=[], > removedBonds=[]), log id: 16707574 >2014-09-24 10:27:48,817 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetupNetworksVDSCommand] (org.ovirt.thread.pool-6-thread-20) [6c53baba] FINISH, SetupNetworksVDSCommand, log id: 16707574 >2014-09-24 10:27:48,821 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-24 10:28:04,203 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-24 10:28:04,441 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) Correlation ID: 6c53baba, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-24 10:28:04,446 INFO [org.ovirt.engine.core.bll.network.host.CommitNetworkChangesCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] Running command: CommitNetworkChangesCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN >2014-09-24 10:28:04,449 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] START, SetSafeNetworkConfigVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 34d04634 >2014-09-24 10:28:04,475 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetSafeNetworkConfigVDSCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] FINISH, SetSafeNetworkConfigVDSCommand, log id: 34d04634 >2014-09-24 10:28:04,494 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [27e641a5] Correlation ID: 27e641a5, Call Stack: null, Custom Event ID: -1, Message: Network changes were saved on host blade >2014-09-24 10:28:04,497 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=Initializing, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5476558a >2014-09-24 10:28:04,519 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] FINISH, SetVdsStatusVDSCommand, log id: 5476558a >2014-09-24 10:28:04,519 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] After Installation host blade, VDS >2014-09-24 10:28:04,541 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-20) [27e641a5] Correlation ID: 1029b7df, Job ID: c441266f-f8f2-4266-a5e9-7a2a2de934b7, Call Stack: null, Custom Event ID: -1, Message: Host blade installed >2014-09-24 10:28:04,552 INFO [org.ovirt.engine.core.bll.InstallVdsInternalCommand] (org.ovirt.thread.pool-6-thread-20) [27e641a5] Lock freed to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:28:07,096 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-17) START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 3396f794 >2014-09-24 10:28:07,221 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-17) FINISH, GetHardwareInfoVDSCommand, log id: 3396f794 >2014-09-24 10:28:07,248 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-17) [2021671b] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,313 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-17) [4a69eec6] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,316 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [4a69eec6] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 17e566de >2014-09-24 10:28:07,343 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [4a69eec6] FINISH, SetVdsStatusVDSCommand, log id: 17e566de >2014-09-24 10:28:07,370 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [4a69eec6] Correlation ID: 4a69eec6, Job ID: 2cd3a337-d7fd-4ac8-acff-742f6ded465b, Call Stack: null, Custom Event ID: -1, Message: Host blade moved to Non-Operational state. >2014-09-24 10:28:07,394 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [4a69eec6] Correlation ID: 2021671b, Call Stack: null, Custom Event ID: -1, Message: Host blade moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-24 10:28:07,452 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-17) [4a69eec6] START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 511249e4 >2014-09-24 10:28:07,460 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-17) [4a69eec6] FINISH, GetHardwareInfoVDSCommand, log id: 511249e4 >2014-09-24 10:28:07,487 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-17) [8a4d777] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,531 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-17) [3a54e897] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,534 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [3a54e897] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 441dc7c6 >2014-09-24 10:28:07,546 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [3a54e897] FINISH, SetVdsStatusVDSCommand, log id: 441dc7c6 >2014-09-24 10:28:07,575 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [3a54e897] Correlation ID: 8a4d777, Call Stack: null, Custom Event ID: -1, Message: Host blade moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-24 10:28:07,589 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [3a54e897] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host blade was set to NonOperational. >2014-09-24 10:28:07,628 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-17) [6d201e7c] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,671 INFO [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-17) [6990bf16] Running command: SetNonOperationalVdsCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,674 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [6990bf16] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=NonOperational, nonOperationalReason=CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER, stopSpmFailureLogged=false), log id: 79a38e4 >2014-09-24 10:28:07,700 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) [6990bf16] FINISH, SetVdsStatusVDSCommand, log id: 79a38e4 >2014-09-24 10:28:07,730 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-17) [6990bf16] Correlation ID: 6d201e7c, Call Stack: null, Custom Event ID: -1, Message: Host blade moved to Non-Operational state as host does not meet the cluster's minimum CPU level. Missing CPU features : model_SandyBridge >2014-09-24 10:28:07,734 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-17) [53bef1c3] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:28:07,737 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-17) [53bef1c3] Host 96620dcc-2509-48fe-841b-85247771c4fb : blade is already in NonOperational status for reason CPU_TYPE_INCOMPATIBLE_WITH_CLUSTER. SetNonOperationalVds command is skipped. >2014-09-24 10:34:30,939 INFO [org.ovirt.engine.core.bll.UpdateVdsGroupCommand] (ajp-/127.0.0.1:8702-7) [abe62cd] Running command: UpdateVdsGroupCommand internal: false. Entities affected : ID: 00000001-0001-0001-0001-000000000399 Type: VdsGroupsAction group EDIT_CLUSTER_CONFIGURATION with role type ADMIN >2014-09-24 10:34:30,999 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-7) [abe62cd] Correlation ID: abe62cd, Call Stack: null, Custom Event ID: -1, Message: Host cluster Default was updated by admin >2014-09-24 10:34:36,513 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-1) [22ca82d1] Lock Acquired to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:34:36,551 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-49) [22ca82d1] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-24 10:34:36,552 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-49) [22ca82d1] Before acquiring lock in order to prevent monitoring for host blade from data-center Default >2014-09-24 10:34:36,552 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-49) [22ca82d1] Lock acquired, from now a monitoring of host will be skipped for host blade from data-center Default >2014-09-24 10:34:36,561 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [22ca82d1] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 5692938f >2014-09-24 10:34:36,582 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-49) [22ca82d1] FINISH, SetVdsStatusVDSCommand, log id: 5692938f >2014-09-24 10:34:36,624 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-49) Activate finished. Lock released. Monitoring can run now for host blade from data-center Default >2014-09-24 10:34:36,638 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-49) Correlation ID: 22ca82d1, Job ID: 65530f1d-8738-4458-b13d-af080c3b9607, Call Stack: null, Custom Event ID: -1, Message: Host blade was activated by admin. >2014-09-24 10:34:36,649 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-49) Lock freed to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:34:38,438 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 6ab5874 >2014-09-24 10:34:38,446 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-46) FINISH, GetHardwareInfoVDSCommand, log id: 6ab5874 >2014-09-24 10:34:38,477 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-46) [5c3d1046] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:34:38,491 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-46) [1d73f0ea] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:34:38,503 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:34:38,522 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: fe32c49 >2014-09-24 10:34:43,913 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0, 83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: fe32c49 >2014-09-24 10:34:43,913 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] Host blade storage connection was succeeded >2014-09-24 10:34:43,924 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-17) START, ConnectStoragePoolVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vdsId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 353e8571 >2014-09-24 10:34:44,734 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-17) FINISH, ConnectStoragePoolVDSCommand, log id: 353e8571 >2014-09-24 10:34:44,755 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] START, SetMOMPolicyParametersVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 498076ca >2014-09-24 10:34:44,897 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] FINISH, SetMOMPolicyParametersVDSCommand, log id: 498076ca >2014-09-24 10:34:44,910 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host blade was set to Up. >2014-09-24 10:34:44,923 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host blade. >2014-09-24 10:34:44,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-46) [43d3a1fb] Correlation ID: 1d73f0ea, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-24 10:34:45,001 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-46) [7827127f] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:35:03,215 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-43) [71cbcd7f] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-24 10:35:03,219 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-43) [71cbcd7f] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 4347966e >2014-09-24 10:35:03,233 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-43) [71cbcd7f] FINISH, SetVdsStatusVDSCommand, log id: 4347966e >2014-09-24 10:35:03,270 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-43) [71cbcd7f] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:35:03,287 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-43) [71cbcd7f] Correlation ID: 71cbcd7f, Job ID: e49db619-0799-49af-aae5-a26f5de00033, Call Stack: null, Custom Event ID: -1, Message: Host blade was switched to Maintenance mode by admin. >2014-09-24 10:35:06,247 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-67) Updated vds status from Preparing for Maintenance to Maintenance in database, vds = 96620dcc-2509-48fe-841b-85247771c4fb : blade >2014-09-24 10:35:06,280 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-24) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: blade. >2014-09-24 10:35:06,280 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-24) Removing vds [96620dcc-2509-48fe-841b-85247771c4fb] from the domain in maintenance cache >2014-09-24 10:35:06,280 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-24) Removing host(s) [96620dcc-2509-48fe-841b-85247771c4fb] from hosts unseen domain report cache >2014-09-24 10:35:06,283 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-67) START, DisconnectStoragePoolVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 2), log id: 5474230e >2014-09-24 10:35:08,334 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, DisconnectStoragePoolVDSCommand, log id: 5474230e >2014-09-24 10:35:08,335 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-67) [451be192] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:35:08,355 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-67) [451be192] START, DisconnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 55123c28 >2014-09-24 10:35:08,554 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-67) [451be192] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0, 83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 55123c28 >2014-09-24 10:35:38,705 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (ajp-/127.0.0.1:8702-1) [152a2768] Lock Acquired to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:35:38,746 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-4) [152a2768] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-24 10:35:38,746 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-4) [152a2768] Before acquiring lock in order to prevent monitoring for host blade from data-center Default >2014-09-24 10:35:38,746 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-4) [152a2768] Lock acquired, from now a monitoring of host will be skipped for host blade from data-center Default >2014-09-24 10:35:38,767 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [152a2768] START, SetVdsStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, status=Unassigned, nonOperationalReason=NONE, stopSpmFailureLogged=false), log id: 4864f175 >2014-09-24 10:35:38,784 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-4) [152a2768] FINISH, SetVdsStatusVDSCommand, log id: 4864f175 >2014-09-24 10:35:38,844 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-4) Activate finished. Lock released. Monitoring can run now for host blade from data-center Default >2014-09-24 10:35:38,853 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-4) Correlation ID: 152a2768, Job ID: a04870dd-cd31-469f-93cd-3b0075755a9e, Call Stack: null, Custom Event ID: -1, Message: Host blade was activated by admin. >2014-09-24 10:35:38,863 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (org.ovirt.thread.pool-6-thread-4) Lock freed to object EngineLock [exclusiveLocks= key: 96620dcc-2509-48fe-841b-85247771c4fb value: VDS >, sharedLocks= ] >2014-09-24 10:35:41,679 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-67) [451be192] START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 61132310 >2014-09-24 10:35:41,686 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-67) [451be192] FINISH, GetHardwareInfoVDSCommand, log id: 61132310 >2014-09-24 10:35:41,726 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler_Worker-67) [4e9cd1d6] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:35:41,739 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (DefaultQuartzScheduler_Worker-67) [678242de] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:35:41,750 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:35:41,768 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 623fba9d >2014-09-24 10:35:47,012 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0, 83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 623fba9d >2014-09-24 10:35:47,012 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] Host blade storage connection was succeeded >2014-09-24 10:35:47,022 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-26) START, ConnectStoragePoolVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vdsId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 3eda39df >2014-09-24 10:35:47,698 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-6-thread-26) FINISH, ConnectStoragePoolVDSCommand, log id: 3eda39df >2014-09-24 10:35:47,736 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] START, SetMOMPolicyParametersVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 4a3fbc04 >2014-09-24 10:35:47,899 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler_Worker-67) [5099506c] FINISH, SetMOMPolicyParametersVDSCommand, log id: 4a3fbc04 >2014-09-24 10:35:47,912 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) [5099506c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Status of host blade was set to Up. >2014-09-24 10:35:47,924 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) [5099506c] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Failed to verify Power Management configuration for Host blade. >2014-09-24 10:35:47,933 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-67) [5099506c] Correlation ID: 678242de, Call Stack: null, Custom Event ID: -1, Message: Unrecognized audit log type has been used. >2014-09-24 10:35:47,966 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-67) [75251210] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-24 10:35:53,361 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (ajp-/127.0.0.1:8702-8) [43640d15] Lock Acquired to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-24 10:35:53,370 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group MANIPULATE_HOST with role type ADMIN >2014-09-24 10:35:53,373 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] START, SetVdsStatusVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, status=PreparingForMaintenance, nonOperationalReason=NONE, stopSpmFailureLogged=true), log id: 80aaa22 >2014-09-24 10:35:53,373 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] VDS rack is spm and moved from up calling resetIrs. >2014-09-24 10:35:53,374 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] START, ResetIrsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, vdsId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, ignoreStopFailed = false), log id: 1f179160 >2014-09-24 10:35:53,377 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] START, SpmStopVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 4bd101e7 >2014-09-24 10:35:53,382 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] SpmStopVDSCommand::Stopping SPM on vds rack, pool id 00000002-0002-0002-0002-0000000001ad >2014-09-24 10:35:54,480 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] FINISH, SpmStopVDSCommand, log id: 4bd101e7 >2014-09-24 10:35:54,489 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] FINISH, ResetIrsVDSCommand, log id: 1f179160 >2014-09-24 10:35:54,504 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] FINISH, SetVdsStatusVDSCommand, log id: 80aaa22 >2014-09-24 10:35:54,507 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] Lock freed to object EngineLock [exclusiveLocks= , sharedLocks= key: 00000002-0002-0002-0002-0000000001ad value: POOL >] >2014-09-24 10:35:54,550 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (org.ovirt.thread.pool-6-thread-28) [43640d15] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDS >2014-09-24 10:35:54,563 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-28) [43640d15] Correlation ID: 43640d15, Job ID: aebcff62-281e-40e7-be41-151724a09bdb, Call Stack: null, Custom Event ID: -1, Message: Host rack was switched to Maintenance mode by admin. >2014-09-24 10:35:55,588 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-59) [62f22e94] Updated vds status from Preparing for Maintenance to Maintenance in database, vds = e58f9fdb-4a02-44ea-be4d-e2162edb495c : rack >2014-09-24 10:35:55,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-2) Clearing cache of pool: 00000002-0002-0002-0002-0000000001ad for problematic entities of VDS: rack. >2014-09-24 10:35:55,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-2) Removing vds [e58f9fdb-4a02-44ea-be4d-e2162edb495c] from the domain in maintenance cache >2014-09-24 10:35:55,626 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-6-thread-2) Removing host(s) [e58f9fdb-4a02-44ea-be4d-e2162edb495c] from hosts unseen domain report cache >2014-09-24 10:35:55,630 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-59) [62f22e94] START, DisconnectStoragePoolVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, vds_spm_id = 1), log id: 6b66b851 >2014-09-24 10:35:56,848 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-33) [2056761] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:35:56,853 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-33) [2056761] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-24 10:35:56,855 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-33) [2056761] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : null >2014-09-24 10:35:56,857 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-33) [2056761] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain fc587b6a-bac3-4668-99de-7f6fd6acce7f status from Active to Unknown, reason : null >2014-09-24 10:35:56,872 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [2056761] Correlation ID: 2056761, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-24 10:35:56,907 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] hostFromVds::selectedVds - blade, spmStatus Free, storage pool Default >2014-09-24 10:35:56,910 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] SPM Init: could not find reported vds or not up - pool:Default vds_spm_id: 1 >2014-09-24 10:35:56,919 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] SPM selection - vds seems as spm rack >2014-09-24 10:35:56,922 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] START, SpmStopVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad), log id: 1b6b3b28 >2014-09-24 10:35:56,930 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] Command org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand return value > >TaskStatusListReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=654, mMessage=Not SPM]] > >2014-09-24 10:35:56,931 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] HostName = rack >2014-09-24 10:35:56,931 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] Command HSMGetAllTasksStatusesVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c) execution failed. Exception: IRSNonOperationalException: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM >2014-09-24 10:35:56,931 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] FINISH, SpmStopVDSCommand, log id: 1b6b3b28 >2014-09-24 10:35:56,932 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] spm stop succeeded, continuing with spm selection >2014-09-24 10:35:56,936 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] starting spm on vds blade, storage pool Default, prevId 1, LVER 6 >2014-09-24 10:35:56,938 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] START, SpmStartVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, prevId=1, prevLVER=6, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=true), log id: 47ba1eb4 >2014-09-24 10:35:56,954 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] spmStart polling started: taskId = 479ba841-53dc-4ae6-a9a9-159b64e4d444 >2014-09-24 10:35:57,793 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (DefaultQuartzScheduler_Worker-59) [62f22e94] FINISH, DisconnectStoragePoolVDSCommand, log id: 6b66b851 >2014-09-24 10:35:57,795 INFO [org.ovirt.engine.core.bll.storage.DisconnectHostFromStoragePoolServersCommand] (DefaultQuartzScheduler_Worker-59) [570a152a] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-24 10:35:57,812 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-59) [570a152a] START, DisconnectStorageServerVDSCommand(HostName = rack, HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4ba8ef37 >2014-09-24 10:35:58,117 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (DefaultQuartzScheduler_Worker-59) [570a152a] FINISH, DisconnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0, 83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 4ba8ef37 >2014-09-24 10:36:18,176 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] spmStart polling ended: taskId = 479ba841-53dc-4ae6-a9a9-159b64e4d444 task status = finished >2014-09-24 10:36:18,183 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] spmStart polling ended, spm status: SPM >2014-09-24 10:36:18,185 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] START, HSMClearTaskVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, taskId=479ba841-53dc-4ae6-a9a9-159b64e4d444), log id: 49c71175 >2014-09-24 10:36:18,193 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] FINISH, HSMClearTaskVDSCommand, log id: 49c71175 >2014-09-24 10:36:18,193 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (DefaultQuartzScheduler_Worker-33) [2056761] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@6b09e79c, log id: 47ba1eb4 >2014-09-24 10:36:18,197 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) [2056761] Initialize Irs proxy from vds: slot-7.rhev.lab.eng.brq.redhat.com >2014-09-24 10:36:18,207 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [2056761] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host blade (Address: slot-7.rhev.lab.eng.brq.redhat.com). >2014-09-24 10:36:18,209 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-24 10:36:18,232 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [2056761] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 162af57c >2014-09-24 10:36:18,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [2056761] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-24 10:36:18,354 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [2056761] START, HSMGetAllTasksInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 48a9383f >2014-09-24 10:36:18,362 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [2056761] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 48a9383f >2014-09-24 10:36:18,362 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-6-thread-25) [2056761] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 162af57c >2014-09-24 10:36:18,362 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-6-thread-25) [2056761] Discovered no tasks on Storage Pool Default >2014-09-24 10:36:46,536 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (ajp-/127.0.0.1:8702-3) [41a6c3e5] Lock Acquired to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-24 10:36:46,549 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] Running command: RemoveVdsCommand internal: false. Entities affected : ID: e58f9fdb-4a02-44ea-be4d-e2162edb495c Type: VDSAction group DELETE_HOST with role type ADMIN >2014-09-24 10:36:46,578 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] START, RemoveVdsVDSCommand( HostId = e58f9fdb-4a02-44ea-be4d-e2162edb495c), log id: 2daf9074 >2014-09-24 10:36:46,578 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] vdsManager::disposing >2014-09-24 10:36:46,579 INFO [org.ovirt.engine.core.vdsbroker.RemoveVdsVDSCommand] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] FINISH, RemoveVdsVDSCommand, log id: 2daf9074 >2014-09-24 10:36:46,586 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] Correlation ID: 41a6c3e5, Call Stack: null, Custom Event ID: -1, Message: Host rack was removed by admin. >2014-09-24 10:36:46,587 INFO [org.ovirt.engine.core.bll.RemoveVdsCommand] (org.ovirt.thread.pool-6-thread-13) [41a6c3e5] Lock freed to object EngineLock [exclusiveLocks= key: e58f9fdb-4a02-44ea-be4d-e2162edb495c value: VDS >, sharedLocks= ] >2014-09-24 10:41:55,526 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-24 10:41:55,526 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-9) Invalid locale found in configuration: >2014-09-24 10:41:57,879 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-24 10:41:57,879 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-10) Invalid locale found in configuration: >2014-09-24 10:42:21,653 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-5) [61a38b61] Lock Acquired to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-24 10:42:21,789 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [61a38b61] START, IsVmDuringInitiatingVDSCommand( vmId = 14034173-04d2-4b26-b4da-42ce14e33e13), log id: 1204600 >2014-09-24 10:42:21,789 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-5) [61a38b61] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 1204600 >2014-09-24 10:42:21,965 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] Running command: RunVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-24 10:42:22,087 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] START, UpdateVmDynamicDataVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@97bdc7a3), log id: 7d33c61e >2014-09-24 10:42:22,100 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] FINISH, UpdateVmDynamicDataVDSCommand, log id: 7d33c61e >2014-09-24 10:42:22,163 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] START, CreateVmVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: e47fbd9 >2014-09-24 10:42:22,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] START, CreateVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, vm=VM [el6.5]), log id: 73165146 >2014-09-24 10:42:22,387 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=14034173-04d2-4b26-b4da-42ce14e33e13,acpiEnable=true,custom={device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447device_65a7ea4f-1bb4-4cb5-88f6-60f9677c4856=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=65a7ea4f-1bb4-4cb5-88f6-60f9677c4856, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=255c24c0-d8b6-4ac7-b227-d917a349b447, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=a04587f1-e816-4426-af38-e89f9611a4d7, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bc=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=ea7ebce5-9b99-47ec-9532-e6d894fb47bc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=5120,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, type=pci, slot=0x02, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=06d56511-11b1-42cb-bf27-371743a81867}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=3f23874c-1813-4b86-a266-fd6e5916fa77}, {shared=false, index=0, volumeID=7172d947-e2c8-4595-80bf-933c22f83b06, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=021e5bea-9f8e-446d-b19c-fb68839d4972, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=021e5bea-9f8e-446d-b19c-fb68839d4972}, {nicModel=pv, address={bus=0x00, domain=0x0000, type=pci, slot=0x03, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:91, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92461e66-f605-47da-8c88-9f857f128b3c}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x07, function=0x0}, specParams={model=virtio}, device=memballoon, type=balloon, deviceId=ce3f7b1c-f540-4d2d-b285-769eb9e05542}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=fbd9cae2-f563-4db3-b236-4e763aa059fe}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=f005d666-1889-4213-b96b-e5f48281f18e}],vmName=el6.5,cpuType=Westmere,fileTransferEnable=true >2014-09-24 10:42:22,453 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] FINISH, CreateVDSCommand, log id: 73165146 >2014-09-24 10:42:22,473 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: e47fbd9 >2014-09-24 10:42:22,474 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-47) [61a38b61] Lock freed to object EngineLock [exclusiveLocks= key: 14034173-04d2-4b26-b4da-42ce14e33e13 value: VM >, sharedLocks= ] >2014-09-24 10:42:22,491 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-47) [61a38b61] Correlation ID: 61a38b61, Job ID: a92f020e-bf53-4aa4-a60a-1e03f0e93b4b, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 was started by admin (Host: blade). >2014-09-24 10:42:22,689 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-34) START, FullListVdsCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 71c6a066 >2014-09-24 10:42:22,703 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-34) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=0, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=-1, spiceSslCipherSuite=DEFAULT, cpuType=Westmere, smp=1, numaTune={mode=preferred}, custom={device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bc=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=ea7ebce5-9b99-47ec-9532-e6d894fb47bc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447device_65a7ea4f-1bb4-4cb5-88f6-60f9677c4856=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=65a7ea4f-1bb4-4cb5-88f6-60f9677c4856, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=255c24c0-d8b6-4ac7-b227-d917a349b447, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=a04587f1-e816-4426-af38-e89f9611a4d7, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=WaitForLaunch, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.63.137, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@274cc2e5, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 71c6a066 >2014-09-24 10:42:22,707 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-34) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.63.137, displayNetwork=rhevm}, deviceType=graphics, type=graphics} >2014-09-24 10:42:25,811 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from WaitForLaunch --> PoweringUp >2014-09-24 10:42:25,818 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-48) START, FullListVdsCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 4a2f0a5d >2014-09-24 10:42:25,835 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-48) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=20148, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5900, spiceSslCipherSuite=DEFAULT, cpuType=Westmere, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bc=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=ea7ebce5-9b99-47ec-9532-e6d894fb47bc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447device_65a7ea4f-1bb4-4cb5-88f6-60f9677c4856=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=65a7ea4f-1bb4-4cb5-88f6-60f9677c4856, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=255c24c0-d8b6-4ac7-b227-d917a349b447, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=a04587f1-e816-4426-af38-e89f9611a4d7, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=Up, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.63.137, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@7dff8dec, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 4a2f0a5d >2014-09-24 10:42:25,838 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-48) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.63.137, displayNetwork=rhevm}, deviceType=graphics, type=graphics, tlsPort=5900} >2014-09-24 10:43:23,999 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-10) VM el6.5 14034173-04d2-4b26-b4da-42ce14e33e13 moved from PoweringUp --> Up >2014-09-24 10:43:24,032 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-10) Correlation ID: 61a38b61, Job ID: a92f020e-bf53-4aa4-a60a-1e03f0e93b4b, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 started on Host blade >2014-09-24 10:44:40,081 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp-/127.0.0.1:8702-11) [162328cd] Lock Acquired to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-24 10:44:40,151 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-11) [162328cd] START, IsVmDuringInitiatingVDSCommand( vmId = 349f8284-643b-4d9e-b208-aff7af2b5fb3), log id: 2da9fe3b >2014-09-24 10:44:40,152 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp-/127.0.0.1:8702-11) [162328cd] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 2da9fe3b >2014-09-24 10:44:40,226 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] Running command: RunVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-24 10:44:40,278 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] START, UpdateVmDynamicDataVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@491d34f3), log id: 1abd50c3 >2014-09-24 10:44:40,287 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] FINISH, UpdateVmDynamicDataVDSCommand, log id: 1abd50c3 >2014-09-24 10:44:40,302 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] START, CreateVmVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 54778199 >2014-09-24 10:44:40,312 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] START, CreateVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, vm=VM [f20]), log id: 719ea60 >2014-09-24 10:44:40,383 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,displayNetwork=rhevm,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3,acpiEnable=true,custom={},spiceSslCipherSuite=DEFAULT,memSize=1024,smp=1,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1024,display=qxl,smartcardEnable=false,bootMenuEnable=false,spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,numaTune={mode=preferred},maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, type=pci, slot=0x02, function=0x0}, specParams={vram=32768, heads=1}, device=qxl, type=video, deviceId=798508b7-1373-4465-a7b4-1c5f25d38bd6}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=fa22ef87-6e17-4c22-b6fe-50d5d4ae9758}, {shared=false, index=0, volumeID=455469db-f6b6-4e6a-b906-8dc04ce3907a, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, type=pci, slot=0x06, function=0x0}, domainID=eb633636-3019-47a9-9f5a-fc81f8dfd7dc, imageID=72c1d2e5-3906-472a-884a-beaf8178a948, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-0000000001ad, readonly=false, deviceId=72c1d2e5-3906-472a-884a-beaf8178a948}, {nicModel=pv, address={bus=0x00, domain=0x0000, type=pci, slot=0x03, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:d0:3f:90, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=rhevm, deviceId=92a2107d-cec5-47a5-96bf-3dd4ec81fb6c}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, type=pci, slot=0x04, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=3838190b-abba-4a39-b5b4-3845d56b684b}, {address={bus=0x00, domain=0x0000, type=pci, slot=0x05, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=1a591966-8fde-4c36-a832-8c75d400b1c1}],vmName=f20,cpuType=Westmere,fileTransferEnable=true >2014-09-24 10:44:40,434 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] FINISH, CreateVDSCommand, log id: 719ea60 >2014-09-24 10:44:40,451 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 54778199 >2014-09-24 10:44:40,451 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-6-thread-40) [162328cd] Lock freed to object EngineLock [exclusiveLocks= key: 349f8284-643b-4d9e-b208-aff7af2b5fb3 value: VM >, sharedLocks= ] >2014-09-24 10:44:40,469 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-40) [162328cd] Correlation ID: 162328cd, Job ID: e0acc0a6-df19-42f1-9ecf-68c3a78dbf27, Call Stack: null, Custom Event ID: -1, Message: VM f20 was started by admin (Host: blade). >2014-09-24 10:44:43,299 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from WaitForLaunch --> PoweringUp >2014-09-24 10:44:43,305 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-77) START, FullListVdsCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb], vmIds=[349f8284-643b-4d9e-b208-aff7af2b5fb3]), log id: 450f9e8c >2014-09-24 10:44:43,334 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-77) FINISH, FullListVdsCommand, return: [{acpiEnable=true, emulatedMachine=rhel6.5.0, pid=20148, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5900, spiceSslCipherSuite=DEFAULT, cpuType=Westmere, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bc=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=ea7ebce5-9b99-47ec-9532-e6d894fb47bc, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=1}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447device_65a7ea4f-1bb4-4cb5-88f6-60f9677c4856=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=65a7ea4f-1bb4-4cb5-88f6-60f9677c4856, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=2}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7device_ea7ebce5-9b99-47ec-9532-e6d894fb47bcdevice_255c24c0-d8b6-4ac7-b227-d917a349b447=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=255c24c0-d8b6-4ac7-b227-d917a349b447, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={bus=0, controller=0, type=virtio-serial, port=3}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_a04587f1-e816-4426-af38-e89f9611a4d7=VmDevice {vmId=14034173-04d2-4b26-b4da-42ce14e33e13, deviceId=a04587f1-e816-4426-af38-e89f9611a4d7, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}}, vmType=kvm, memSize=5120, smpCoresPerSocket=1, vmName=el6.5, nice=0, status=Up, bootMenuEnable=false, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, copyPasteEnable=true, displayIp=10.34.63.137, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@4721cc0, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}, {acpiEnable=true, emulatedMachine=rhel6.5.0, pid=22686, memGuaranteedSize=1024, transparentHugePages=true, displaySecurePort=5901, spiceSslCipherSuite=DEFAULT, cpuType=Westmere, smp=1, pauseCode=NOERR, numaTune={mode=preferred}, custom={}, vmType=kvm, memSize=1024, smpCoresPerSocket=1, vmName=f20, nice=0, status=Up, bootMenuEnable=false, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, copyPasteEnable=true, displayIp=10.34.63.137, displayPort=-1, smartcardEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, fileTransferEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, kvmEnable=true, displayNetwork=rhevm, devices=[Ljava.lang.Object;@5fbd8387, timeOffset=0, maxVCpus=16, clientIp=, display=qxl}], log id: 450f9e8c >2014-09-24 10:44:43,337 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Received a spice Device without an address when processing VM 14034173-04d2-4b26-b4da-42ce14e33e13 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.63.137, displayNetwork=rhevm}, deviceType=graphics, type=graphics, tlsPort=5900} >2014-09-24 10:44:43,343 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Received a memballoon Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=memballoon, specParams={model=none}, deviceType=balloon, type=balloon, alias=balloon0} >2014-09-24 10:44:43,343 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-77) Received a spice Device without an address when processing VM 349f8284-643b-4d9e-b208-aff7af2b5fb3 devices, skipping device: {device=spice, specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, copyPasteEnable=true, keyMap=en-us, displayIp=10.34.63.137, displayNetwork=rhevm}, deviceType=graphics, type=graphics, tlsPort=5901} >2014-09-24 10:45:41,601 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-33) [2056761] VM f20 349f8284-643b-4d9e-b208-aff7af2b5fb3 moved from PoweringUp --> Up >2014-09-24 10:45:41,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-33) [2056761] Correlation ID: 162328cd, Job ID: e0acc0a6-df19-42f1-9ecf-68c3a78dbf27, Call Stack: null, Custom Event ID: -1, Message: VM f20 started on Host blade >2014-09-24 11:05:27,590 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-42) Principal admin::internal synchronized >2014-09-24 11:05:34,036 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-24 11:10:14,237 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-24 11:10:14,240 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] Entered (VM f20). >2014-09-24 11:10:14,240 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] Sending shutdown command for VM f20. >2014-09-24 11:10:14,245 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] START, DestroyVmVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 23195cab >2014-09-24 11:10:14,304 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] START, DestroyVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=30, gracefully=true, reason=), log id: 62f6c6d5 >2014-09-24 11:10:14,313 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] FINISH, DestroyVDSCommand, log id: 62f6c6d5 >2014-09-24 11:10:14,336 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 23195cab >2014-09-24 11:10:14,361 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-18) [5e5b6de7] Correlation ID: 5e5b6de7, Job ID: ddf150ab-778f-4e76-87ea-610f0d42bf60, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM f20 (Host: blade) (Reason: Not Specified). >2014-09-24 11:10:17,229 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-9) [78b8456] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-24 11:10:17,246 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [78b8456] START, SetVmTicketVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, ticket=uep48BJU2PAi, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 1d4664c4 >2014-09-24 11:10:17,320 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-9) [78b8456] FINISH, SetVmTicketVDSCommand, log id: 1d4664c4 >2014-09-24 11:10:17,344 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-9) [78b8456] Correlation ID: 78b8456, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM f20 >2014-09-24 11:10:19,943 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-74) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM f20. >2014-09-24 11:10:26,062 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM f20. >2014-09-24 11:10:27,197 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5246a2ab >2014-09-24 11:10:27,198 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-14) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2b1e694b >2014-09-24 11:10:27,235 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-19) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 5246a2ab >2014-09-24 11:10:27,244 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-19) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 11:10:27,266 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-14) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 2b1e694b >2014-09-24 11:10:27,288 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-14) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 11:10:27,305 INFO [org.ovirt.engine.core.bll.SetVmTicketCommand] (ajp-/127.0.0.1:8702-1) [7678f20c] Running command: SetVmTicketCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group CONNECT_TO_VM with role type USER >2014-09-24 11:10:27,324 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [7678f20c] START, SetVmTicketVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, ticket=2nTc6hDU9N+j, validTime=120,m userName=admin, userId=fdfc627c-d875-11e0-90f0-83df133b58cc), log id: 748191c >2014-09-24 11:10:27,351 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetVmTicketVDSCommand] (ajp-/127.0.0.1:8702-1) [7678f20c] FINISH, SetVmTicketVDSCommand, log id: 748191c >2014-09-24 11:10:27,371 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) [7678f20c] Correlation ID: 7678f20c, Call Stack: null, Custom Event ID: -1, Message: user admin initiated console session for VM el6.5 >2014-09-24 11:10:29,180 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-95) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin is connected to VM el6.5. >2014-09-24 11:10:32,264 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin got disconnected from VM el6.5. >2014-09-24 11:10:33,409 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] Running command: ShutdownVmCommand internal: false. Entities affected : ID: 14034173-04d2-4b26-b4da-42ce14e33e13 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-24 11:10:33,412 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] Entered (VM el6.5). >2014-09-24 11:10:33,412 INFO [org.ovirt.engine.core.bll.ShutdownVmCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] Sending shutdown command for VM el6.5. >2014-09-24 11:10:33,415 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] START, DestroyVmVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=30, gracefully=true, reason=), log id: 418d86c2 >2014-09-24 11:10:33,425 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] START, DestroyVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=30, gracefully=true, reason=), log id: 43daf33f >2014-09-24 11:10:33,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] FINISH, DestroyVDSCommand, log id: 43daf33f >2014-09-24 11:10:33,454 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-43) [a117e89] FINISH, DestroyVmVDSCommand, return: PoweringDown, log id: 418d86c2 >2014-09-24 11:10:33,477 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-43) [a117e89] Correlation ID: a117e89, Job ID: 2d8d02bf-54b2-4bda-bf13-1672dbb5075c, Call Stack: null, Custom Event ID: -1, Message: VM shutdown initiated by admin on VM el6.5 (Host: blade) (Reason: Not Specified). >2014-09-24 11:10:38,563 INFO [org.ovirt.engine.core.bll.StopVmCommand] (org.ovirt.thread.pool-6-thread-3) [746434f8] Running command: StopVmCommand internal: false. Entities affected : ID: 349f8284-643b-4d9e-b208-aff7af2b5fb3 Type: VMAction group VM_BASIC_OPERATIONS with role type USER >2014-09-24 11:10:38,571 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-3) [746434f8] START, DestroyVmVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=0, gracefully=false, reason=), log id: 17b561f8 >2014-09-24 11:10:38,580 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-3) [746434f8] START, DestroyVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=349f8284-643b-4d9e-b208-aff7af2b5fb3, force=false, secondsToWait=0, gracefully=false, reason=), log id: 181f59de >2014-09-24 11:10:38,947 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (org.ovirt.thread.pool-6-thread-3) [746434f8] FINISH, DestroyVDSCommand, log id: 181f59de >2014-09-24 11:10:38,966 INFO [org.ovirt.engine.core.vdsbroker.DestroyVmVDSCommand] (org.ovirt.thread.pool-6-thread-3) [746434f8] FINISH, DestroyVmVDSCommand, return: Down, log id: 17b561f8 >2014-09-24 11:10:38,978 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-42) [4380b5c1] Running command: ProcessDownVmCommand internal: true. >2014-09-24 11:10:38,996 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-3) [746434f8] Correlation ID: 746434f8, Job ID: 8bd302e7-20d7-4883-aa74-e14097e9ad7b, Call Stack: null, Custom Event ID: -1, Message: VM f20 powered off by admin (Host: blade) (Reason: Not Specified). >2014-09-24 11:10:50,624 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-1) START, DestroyVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vmId=14034173-04d2-4b26-b4da-42ce14e33e13, force=false, secondsToWait=0, gracefully=false, reason=), log id: 6889173d >2014-09-24 11:10:50,631 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, DestroyVDSCommand, log id: 6889173d >2014-09-24 11:10:50,653 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM el6.5 is down. Exit message: Admin shut down from the engine >2014-09-24 11:10:50,656 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) VM el6.5 (14034173-04d2-4b26-b4da-42ce14e33e13) is running in db and not running in VDS blade >2014-09-24 11:10:50,656 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) START, FullListVdsCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb], vmIds=[14034173-04d2-4b26-b4da-42ce14e33e13]), log id: 75f1404d >2014-09-24 11:10:50,662 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-1) FINISH, FullListVdsCommand, return: [], log id: 75f1404d >2014-09-24 11:10:50,679 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (org.ovirt.thread.pool-6-thread-36) [630d4107] Running command: ProcessDownVmCommand internal: true. >2014-09-24 12:05:27,602 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-60) Principal admin::internal synchronized >2014-09-24 12:05:34,066 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-37) Attempting to update VMs/Templates Ovf. >2014-09-24 12:10:27,299 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7363b118 >2014-09-24 12:10:27,299 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-41) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5036bd89 >2014-09-24 12:10:27,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 7363b118 >2014-09-24 12:10:27,346 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-13) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 12:10:27,373 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-41) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 5036bd89 >2014-09-24 12:10:27,383 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-41) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 13:05:27,614 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-68) Principal admin::internal synchronized >2014-09-24 13:05:34,092 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-80) Attempting to update VMs/Templates Ovf. >2014-09-24 13:10:27,392 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 679fec7f >2014-09-24 13:10:27,393 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-10) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 468a6f99 >2014-09-24 13:10:27,426 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 679fec7f >2014-09-24 13:10:27,437 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-43) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 13:10:27,460 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-10) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 468a6f99 >2014-09-24 13:10:27,466 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-10) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 14:05:27,638 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-70) Principal admin::internal synchronized >2014-09-24 14:05:34,121 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-52) Attempting to update VMs/Templates Ovf. >2014-09-24 14:10:27,478 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 635a2544 >2014-09-24 14:10:27,478 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-32) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 68380a22 >2014-09-24 14:10:27,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 635a2544 >2014-09-24 14:10:27,527 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-4) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 14:10:27,557 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-32) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 68380a22 >2014-09-24 14:10:27,565 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-32) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 15:05:27,648 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-24 15:05:34,146 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-57) Attempting to update VMs/Templates Ovf. >2014-09-24 15:10:27,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) [3775fd20] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4f40260d >2014-09-24 15:10:27,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) [3775fd20] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4d4d6cbc >2014-09-24 15:10:27,608 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) [3775fd20] FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 4f40260d >2014-09-24 15:10:27,619 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-15) [3775fd20] Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 15:10:27,646 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) [3775fd20] FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 4d4d6cbc >2014-09-24 15:10:27,655 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-26) [3775fd20] Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 16:05:27,659 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-98) [153916cb] Principal admin::internal synchronized >2014-09-24 16:05:34,177 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-24 16:10:27,666 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 559bd5cf >2014-09-24 16:10:27,667 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-1) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1c71f08e >2014-09-24 16:10:27,702 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 559bd5cf >2014-09-24 16:10:27,712 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-30) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 16:10:27,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-1) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 1c71f08e >2014-09-24 16:10:27,741 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-1) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 17:05:27,669 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-44) Principal admin::internal synchronized >2014-09-24 17:05:34,229 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-09-24 17:10:27,753 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 62d777cb >2014-09-24 17:10:27,757 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4690cdea >2014-09-24 17:10:27,792 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 62d777cb >2014-09-24 17:10:27,802 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-11) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 17:10:27,826 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 4690cdea >2014-09-24 17:10:27,835 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-3) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 18:05:27,680 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-99) Principal admin::internal synchronized >2014-09-24 18:05:34,257 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-24 18:10:27,842 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 35793306 >2014-09-24 18:10:27,843 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5ab50a47 >2014-09-24 18:10:27,875 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 35793306 >2014-09-24 18:10:27,883 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-24) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 18:10:27,905 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 5ab50a47 >2014-09-24 18:10:27,914 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-48) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 19:05:27,690 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-6) Principal admin::internal synchronized >2014-09-24 19:05:34,294 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-1) Attempting to update VMs/Templates Ovf. >2014-09-24 19:10:27,922 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-42) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 706c3cbe >2014-09-24 19:10:27,923 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1145eb30 >2014-09-24 19:10:27,959 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-42) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 706c3cbe >2014-09-24 19:10:27,968 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-42) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 19:10:28,010 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 1145eb30 >2014-09-24 19:10:28,019 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-50) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 20:05:27,699 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-61) Principal admin::internal synchronized >2014-09-24 20:05:34,324 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Attempting to update VMs/Templates Ovf. >2014-09-24 20:10:28,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-36) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 344f6267 >2014-09-24 20:10:28,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 47c57305 >2014-09-24 20:10:28,067 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-36) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 344f6267 >2014-09-24 20:10:28,075 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-36) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 20:10:28,098 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 47c57305 >2014-09-24 20:10:28,106 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-43) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 21:05:27,710 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-87) Principal admin::internal synchronized >2014-09-24 21:05:34,348 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-76) Attempting to update VMs/Templates Ovf. >2014-09-24 21:10:28,115 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 3443e32 >2014-09-24 21:10:28,119 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: af61732 >2014-09-24 21:10:28,152 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 3443e32 >2014-09-24 21:10:28,162 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-4) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 21:10:28,187 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-13) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: af61732 >2014-09-24 21:10:28,195 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-13) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 22:05:27,722 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-36) Principal admin::internal synchronized >2014-09-24 22:05:34,373 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-40) Attempting to update VMs/Templates Ovf. >2014-09-24 22:10:28,205 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-39) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5dc3b4d2 >2014-09-24 22:10:28,207 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2610d00a >2014-09-24 22:10:28,255 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-39) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 5dc3b4d2 >2014-09-24 22:10:28,266 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-39) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 22:10:28,294 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-15) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 2610d00a >2014-09-24 22:10:28,300 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-15) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 23:05:27,734 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-49) Principal admin::internal synchronized >2014-09-24 23:05:34,400 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-71) Attempting to update VMs/Templates Ovf. >2014-09-24 23:10:28,314 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 176cd59 >2014-09-24 23:10:28,317 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 29b6c3c8 >2014-09-24 23:10:28,352 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 176cd59 >2014-09-24 23:10:28,360 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-33) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-24 23:10:28,386 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-30) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 29b6c3c8 >2014-09-24 23:10:28,393 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-30) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 00:05:27,745 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-6) Principal admin::internal synchronized >2014-09-25 00:05:34,427 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-58) Attempting to update VMs/Templates Ovf. >2014-09-25 00:10:28,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-18) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 78253c81 >2014-09-25 00:10:28,401 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-3) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 3e66ec95 >2014-09-25 00:10:28,436 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-18) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 78253c81 >2014-09-25 00:10:28,443 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-18) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 00:10:28,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-3) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 3e66ec95 >2014-09-25 00:10:28,475 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-3) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 01:05:27,755 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-41) Principal admin::internal synchronized >2014-09-25 01:05:34,451 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-61) Attempting to update VMs/Templates Ovf. >2014-09-25 01:10:28,482 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7d97ede4 >2014-09-25 01:10:28,485 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4de233f9 >2014-09-25 01:10:28,517 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 7d97ede4 >2014-09-25 01:10:28,526 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-24) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 01:10:28,552 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-48) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 4de233f9 >2014-09-25 01:10:28,562 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-48) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 02:05:27,767 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-58) Principal admin::internal synchronized >2014-09-25 02:05:34,479 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-25 02:10:28,572 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2d6fa997 >2014-09-25 02:10:28,573 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-41) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4cbeddf5 >2014-09-25 02:10:28,609 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 2d6fa997 >2014-09-25 02:10:28,616 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-50) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 02:10:28,643 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-41) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 4cbeddf5 >2014-09-25 02:10:28,651 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-41) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 03:05:27,779 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-7) [31de4b0a] Principal admin::internal synchronized >2014-09-25 03:05:34,504 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-50) Attempting to update VMs/Templates Ovf. >2014-09-25 03:10:28,658 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1c0fdc59 >2014-09-25 03:10:28,659 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 48f18cc4 >2014-09-25 03:10:28,702 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 1c0fdc59 >2014-09-25 03:10:28,712 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-23) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 03:10:28,743 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-43) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 48f18cc4 >2014-09-25 03:10:28,752 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-43) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 03:35:35,001 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-69) Start deleteAgedOutAuditLogs >2014-09-25 03:35:35,006 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-69) Finished deleteAgedOutAuditLogs >2014-09-25 03:35:35,007 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-49) Start deleteAgedOutCommandEntities >2014-09-25 03:35:35,011 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (DefaultQuartzScheduler_Worker-49) Finished deleteAgedOutCommandEntities >2014-09-25 04:05:27,790 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-35) Principal admin::internal synchronized >2014-09-25 04:05:34,528 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-94) Attempting to update VMs/Templates Ovf. >2014-09-25 04:10:28,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5344e1a5 >2014-09-25 04:10:28,764 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-22) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 79d9c80a >2014-09-25 04:10:28,803 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 5344e1a5 >2014-09-25 04:10:28,813 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-4) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 04:10:28,838 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-22) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 79d9c80a >2014-09-25 04:10:28,846 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-22) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 05:05:27,804 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-50) Principal admin::internal synchronized >2014-09-25 05:05:34,555 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-18) Attempting to update VMs/Templates Ovf. >2014-09-25 05:10:28,854 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-44) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: e19b757 >2014-09-25 05:10:28,855 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4e541f4e >2014-09-25 05:10:28,889 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-44) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: e19b757 >2014-09-25 05:10:28,899 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-44) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 05:10:28,930 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-35) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 4e541f4e >2014-09-25 05:10:28,938 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-35) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 06:05:27,819 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-10) Principal admin::internal synchronized >2014-09-25 06:05:34,582 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-7) [31de4b0a] Attempting to update VMs/Templates Ovf. >2014-09-25 06:10:28,946 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 688c692d >2014-09-25 06:10:28,948 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-6) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 561bd67 >2014-09-25 06:10:28,983 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 688c692d >2014-09-25 06:10:28,994 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-33) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 06:10:29,013 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-6) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 561bd67 >2014-09-25 06:10:29,022 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-6) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 07:05:27,830 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-67) [75251210] Principal admin::internal synchronized >2014-09-25 07:05:34,608 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-74) Attempting to update VMs/Templates Ovf. >2014-09-25 07:10:29,029 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-18) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 28f9d2f9 >2014-09-25 07:10:29,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1c9a9d78 >2014-09-25 07:10:29,068 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-18) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 28f9d2f9 >2014-09-25 07:10:29,075 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-18) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 07:10:29,104 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 1c9a9d78 >2014-09-25 07:10:29,111 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-34) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 08:05:27,840 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-36) Principal admin::internal synchronized >2014-09-25 08:05:34,634 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-75) Attempting to update VMs/Templates Ovf. >2014-09-25 08:10:29,119 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-20) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 655070f5 >2014-09-25 08:10:29,120 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 647377e7 >2014-09-25 08:10:29,152 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-20) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 655070f5 >2014-09-25 08:10:29,162 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-20) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 08:10:29,189 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 647377e7 >2014-09-25 08:10:29,197 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-24) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 09:05:27,852 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-59) [570a152a] Principal admin::internal synchronized >2014-09-25 09:05:34,662 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-4) [1da6da9d] Attempting to update VMs/Templates Ovf. >2014-09-25 09:10:29,204 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-42) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2ce81035 >2014-09-25 09:10:29,204 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 37801e9b >2014-09-25 09:10:29,240 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-42) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 2ce81035 >2014-09-25 09:10:29,248 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-42) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 09:10:29,275 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 37801e9b >2014-09-25 09:10:29,284 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-11) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 10:05:27,863 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-11) Principal admin::internal synchronized >2014-09-25 10:05:34,692 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-69) Attempting to update VMs/Templates Ovf. >2014-09-25 10:10:29,291 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-5) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 35201374 >2014-09-25 10:10:29,292 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 518f84b9 >2014-09-25 10:10:29,353 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-5) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 35201374 >2014-09-25 10:10:29,363 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-5) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 10:10:29,449 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 518f84b9 >2014-09-25 10:10:29,457 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-23) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 11:05:27,875 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-90) Principal admin::internal synchronized >2014-09-25 11:05:34,720 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-29) [3775fd20] Attempting to update VMs/Templates Ovf. >2014-09-25 11:10:29,463 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) [3db454f1] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 61d1f944 >2014-09-25 11:10:29,466 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) [3db454f1] START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7c5c206c >2014-09-25 11:10:29,500 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-50) [3db454f1] FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 61d1f944 >2014-09-25 11:10:29,508 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-50) [3db454f1] Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 11:10:29,534 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-4) [3db454f1] FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 7c5c206c >2014-09-25 11:10:29,542 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-4) [3db454f1] Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 12:05:27,885 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-67) [75251210] Principal admin::internal synchronized >2014-09-25 12:05:34,749 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-20) Attempting to update VMs/Templates Ovf. >2014-09-25 12:10:29,550 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-44) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5803b40 >2014-09-25 12:10:29,553 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-29) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 6d91bdec >2014-09-25 12:10:29,587 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-44) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 5803b40 >2014-09-25 12:10:29,596 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-44) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 12:10:29,622 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-29) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 6d91bdec >2014-09-25 12:10:29,632 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-29) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 13:05:27,900 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-13) Principal admin::internal synchronized >2014-09-25 13:05:34,774 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-31) Attempting to update VMs/Templates Ovf. >2014-09-25 13:10:29,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 1dc7c975 >2014-09-25 13:10:29,644 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-2) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 277b2f99 >2014-09-25 13:10:29,682 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-33) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 1dc7c975 >2014-09-25 13:10:29,692 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-33) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 13:10:29,716 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-2) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 277b2f99 >2014-09-25 13:10:29,725 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-2) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 14:05:27,910 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-46) [7827127f] Principal admin::internal synchronized >2014-09-25 14:05:34,802 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-30) [3d8a45b] Attempting to update VMs/Templates Ovf. >2014-09-25 14:10:29,732 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7cb69df0 >2014-09-25 14:10:29,733 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-45) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 5eba9d6b >2014-09-25 14:10:29,777 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-34) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 7cb69df0 >2014-09-25 14:10:29,787 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-34) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 14:10:29,809 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-45) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 5eba9d6b >2014-09-25 14:10:29,816 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-45) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 15:05:27,919 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-84) Principal admin::internal synchronized >2014-09-25 15:05:34,830 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-72) Attempting to update VMs/Templates Ovf. >2014-09-25 15:10:29,826 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 4243f85f >2014-09-25 15:10:29,829 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-20) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 34175267 >2014-09-25 15:10:29,859 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-24) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 4243f85f >2014-09-25 15:10:29,868 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-24) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 15:10:29,889 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-20) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 34175267 >2014-09-25 15:10:29,895 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-20) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 16:05:27,930 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-83) Principal admin::internal synchronized >2014-09-25 16:05:34,870 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-64) Attempting to update VMs/Templates Ovf. >2014-09-25 16:10:29,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7f925cd1 >2014-09-25 16:10:29,904 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 620bdec5 >2014-09-25 16:10:29,942 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-26) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 7f925cd1 >2014-09-25 16:10:29,950 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-26) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 16:10:29,976 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-11) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 620bdec5 >2014-09-25 16:10:29,985 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-11) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 17:05:27,939 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-25) Principal admin::internal synchronized >2014-09-25 17:05:34,898 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (DefaultQuartzScheduler_Worker-33) [2056761] Attempting to update VMs/Templates Ovf. >2014-09-25 17:10:29,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-38) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 34654df3 >2014-09-25 17:10:29,990 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) START, GetFileStatsVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 270dcb00 >2014-09-25 17:10:30,026 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-38) FINISH, GetFileStatsVDSCommand, return: {virtio-win_x86.vfd={status=0, ctime=1410269074.0, size=2949120}, virtio-win_amd64.vfd={status=0, ctime=1410269074.0, size=2949120}}, log id: 34654df3 >2014-09-25 17:10:30,033 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-38) Finished automatic refresh process for Floppy file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 17:10:30,110 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetFileStatsVDSCommand] (org.ovirt.thread.pool-6-thread-23) FINISH, GetFileStatsVDSCommand, return: {rhev-tools-setup.iso={status=0, ctime=1410269074.0, size=350183424}, virtio-win.iso={status=0, ctime=1410269074.0, size=149004288}}, log id: 270dcb00 >2014-09-25 17:10:30,119 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (org.ovirt.thread.pool-6-thread-23) Finished automatic refresh process for ISO file type with success, for storage domain id fc587b6a-bac3-4668-99de-7f6fd6acce7f. >2014-09-25 18:01:29,064 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-25 18:01:29,064 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-25 18:01:29,065 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-25 18:01:29,073 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-25 18:01:29,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-25 18:01:29,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-25 18:01:29,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-25 18:01:29,074 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-25 18:01:29,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-25 18:01:29,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-25 18:01:29,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-25 18:01:29,075 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-25 18:01:29,076 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-25 18:01:29,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-25 18:01:29,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-25 18:01:29,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-25 18:01:29,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-25 18:01:29,077 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-25 18:01:29,086 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-25 18:01:29,086 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-25 18:01:29,086 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-25 18:01:29,086 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-25 18:01:29,087 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-25 18:01:29,088 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-25 18:01:29,089 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-25 18:01:29,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-25 18:01:29,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-25 18:01:29,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-09-25 18:01:29,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-09-25 18:01:29,090 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-09-25 18:01:29,098 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-09-25 18:01:29,098 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-25 18:01:29,099 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-25 18:01:29,100 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-25 18:01:29,100 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-25 18:01:29,100 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-25 18:01:29,100 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-25 18:01:29,104 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-25 18:01:40,981 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 43) Start initializing Backend >2014-09-25 18:01:41,358 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 43) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-25 18:01:41,485 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 43) Running ovirt-engine 3.5.0-0.13.beta.el6ev >2014-09-25 18:01:41,486 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 43) Start initializing dictionaries >2014-09-25 18:01:41,494 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 43) Finished initializing dictionaries >2014-09-25 18:01:41,494 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Start initializing AuditLogCleanupManager >2014-09-25 18:01:41,495 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-25 18:01:41,516 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Finished initializing AuditLogCleanupManager >2014-09-25 18:01:41,516 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Start initializing CommandEntityCleanupManager >2014-09-25 18:01:41,516 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-25 18:01:41,518 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 43) Finished initializing CommandEntityCleanupManager >2014-09-25 18:01:41,520 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 43) Start initializing TagsDirector >2014-09-25 18:01:41,531 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 43) Tag root added to tree >2014-09-25 18:01:41,539 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 43) Finished initializing TagsDirector >2014-09-25 18:01:41,540 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 43) Start initializing IsoDomainListSyncronizer >2014-09-25 18:01:41,547 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 43) Finished initializing IsoDomainListSyncronizer >2014-09-25 18:01:41,625 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 43) Completed initializing handlers >2014-09-25 18:01:41,657 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 43) Start initializing ErrorTranslatorImpl >2014-09-25 18:01:41,677 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 43) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-25 18:01:41,679 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 43) Finished initializing ErrorTranslatorImpl >2014-09-25 18:01:41,679 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 43) Start initializing ErrorTranslatorImpl >2014-09-25 18:01:41,679 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 43) Finished initializing ErrorTranslatorImpl >2014-09-25 18:01:41,680 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 43) Mark incomplete jobs as UNKNOWN >2014-09-25 18:01:41,717 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 43) Start initializing JobRepositoryCleanupManager >2014-09-25 18:01:41,717 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 43) Finished initializing JobRepositoryCleanupManager >2014-09-25 18:01:41,718 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 43) Start initializing AutoRecoveryManager >2014-09-25 18:01:41,719 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 43) Finished initializing AutoRecoveryManager >2014-09-25 18:01:41,720 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 43) Start initializing ExecutionMessageDirector >2014-09-25 18:01:41,722 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 43) Finished initializing ExecutionMessageDirector >2014-09-25 18:01:41,766 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Loading extension 'builtin-authn-internal' >2014-09-25 18:01:41,767 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Extension 'builtin-authn-internal' loaded >2014-09-25 18:01:41,769 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Loading extension 'internal' >2014-09-25 18:01:41,773 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Extension 'internal' loaded >2014-09-25 18:01:41,774 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Initializing extension 'builtin-authn-internal' >2014-09-25 18:01:41,774 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Extension 'builtin-authn-internal' initialized >2014-09-25 18:01:41,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Initializing extension 'internal' >2014-09-25 18:01:41,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Extension 'internal' initialized >2014-09-25 18:01:41,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Start of enabled extensions list >2014-09-25 18:01:41,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-25 18:01:41,775 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-25 18:01:41,776 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 43) End of enabled extensions list >2014-09-25 18:01:41,865 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 43) Start initializing DbUserCacheManager >2014-09-25 18:01:41,871 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 43) Finished initializing DbUserCacheManager >2014-09-25 18:01:41,902 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 43) Initialization of AsyncTaskManager completed successfully. >2014-09-25 18:01:41,906 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 43) Start initializing ResourceManager >2014-09-25 18:01:41,946 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-25 18:01:42,001 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 43) Entered VdsManager constructor >2014-09-25 18:01:42,045 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 43) Initialize vdsBroker (slot-7.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-25 18:01:42,233 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 43) VDS 96620dcc-2509-48fe-841b-85247771c4fb was added to the Resource Manager >2014-09-25 18:01:42,269 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 43) Finished initializing ResourceManager >2014-09-25 18:01:42,277 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 43) Initialization of OvfDataUpdater completed successfully. >2014-09-25 18:01:42,277 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Start scheduling to enable vds load balancer >2014-09-25 18:01:42,278 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Finished scheduling to enable vds load balancer >2014-09-25 18:01:42,278 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Start HA Reservation check >2014-09-25 18:01:42,279 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Finished HA Reservation check >2014-09-25 18:01:42,289 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 43) Init VM custom properties utilities >2014-09-25 18:01:42,289 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 43) Init device custom properties utilities >2014-09-25 18:01:42,292 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-09-25 18:01:42,303 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Initializing Scheduling manager >2014-09-25 18:01:42,321 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 63 >2014-09-25 18:01:42,345 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) External scheduler disabled, discovery skipped >2014-09-25 18:01:42,345 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 43) Initialized Scheduling manager >2014-09-25 18:01:42,345 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 43) Initializing DWH Heart Beat >2014-09-25 18:01:42,348 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 43) DWH Heart Beat initialized >2014-09-25 18:01:45,326 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-25 18:01:45,823 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 10d65e73 >2014-09-25 18:01:45,832 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 10d65e73 >2014-09-25 18:01:46,069 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: blade >2014-09-25 18:01:46,237 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [47149f7d] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-25 18:01:46,292 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-4) domain fc587b6a-bac3-4668-99de-7f6fd6acce7f:ISO_DOMAIN in problem. vds: blade >2014-09-25 18:01:52,327 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-25 18:01:52,335 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-25 18:01:52,346 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain fc587b6a-bac3-4668-99de-7f6fd6acce7f status from Active to Unknown, reason : null >2014-09-25 18:01:52,347 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : null >2014-09-25 18:01:52,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-25 18:01:52,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-25 18:01:52,392 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-25 18:01:52,393 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-25 18:01:52,393 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-25 18:01:52,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,394 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,401 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,402 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,403 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-25 18:01:52,852 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Correlation ID: 491f0fdf, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-25 18:01:52,957 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [491f0fdf] hostFromVds::selectedVds - blade, spmStatus SPM, storage pool Default >2014-09-25 18:01:52,964 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Initialize Irs proxy from vds: slot-7.rhev.lab.eng.brq.redhat.com >2014-09-25 18:01:52,972 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [491f0fdf] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host blade (Address: slot-7.rhev.lab.eng.brq.redhat.com). >2014-09-25 18:01:52,975 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-25 18:01:53,015 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 7361f43f >2014-09-25 18:01:53,173 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-25 18:01:53,184 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] START, HSMGetAllTasksInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 7b35db4a >2014-09-25 18:01:53,193 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 7b35db4a >2014-09-25 18:01:53,193 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 7361f43f >2014-09-25 18:01:53,193 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-7) [491f0fdf] Discovered no tasks on Storage Pool Default >2014-09-25 18:05:12,449 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf". >2014-09-25 18:05:12,449 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) The file "/etc/ovirt-engine/engine.conf" doesn't exist or isn't readable. Will return an empty set of properties. >2014-09-25 18:05:12,450 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-rhevm-doc.conf". >2014-09-25 18:05:12,450 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-database.conf". >2014-09-25 18:05:12,451 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf". >2014-09-25 18:05:12,462 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf". >2014-09-25 18:05:12,463 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf". >2014-09-25 18:05:12,463 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/20-branding-rhev.conf". >2014-09-25 18:05:12,463 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Loaded file "/etc/ovirt-engine/engine.conf.d/50-redhat-support-plugin.conf". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_ENABLED" is "true". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_AJP_PORT" is "8702". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_APPS" is "engine.ear restapi.war legacy_restapi.war "/usr/share/ovirt-engine/branding/rhev-2.brand/applications/rhev.ear" redhat-support-plugin-rhev.ear". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_CACHE" is "/var/cache/ovirt-engine". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CHECK_INTERVAL" is "1000". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_CONNECTION_TIMEOUT" is "300000". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DATABASE" is "engine". >2014-09-25 18:05:12,464 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_DRIVER" is "org.postgresql.Driver". >2014-09-25 18:05:12,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_HOST" is "localhost". >2014-09-25 18:05:12,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MAX_CONNECTIONS" is "100". >2014-09-25 18:05:12,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_MIN_CONNECTIONS" is "1". >2014-09-25 18:05:12,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PASSWORD" is "***". >2014-09-25 18:05:12,465 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_PORT" is "5432". >2014-09-25 18:05:12,480 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED" is "False". >2014-09-25 18:05:12,480 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_SECURED_VALIDATION" is "False". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_URL" is "jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DB_USER" is "engine". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DEBUG_ADDRESS" is "". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_DOC" is "/usr/share/doc/ovirt-engine". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_ETC" is "/etc/ovirt-engine". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_EXTENSION_PATH" is "/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_FQDN" is "pb-rh35.rhev.lab.eng.brq.redhat.com". >2014-09-25 18:05:12,481 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_GROUP" is "ovirt". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MAX" is "1g". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HEAP_MIN" is "1g". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_ENABLED" is "false". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PORT" is "None". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTPS_PROTOCOLS" is "SSLv3,TLSv1,TLSv1.1,TLSv1.2". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_ENABLED" is "false". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_HTTP_PORT" is "None". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JAVA_MODULEPATH" is "/usr/share/ovirt-engine/modules". >2014-09-25 18:05:12,482 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_JVM_ARGS" is " -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"". >2014-09-25 18:05:12,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG" is "/var/log/ovirt-engine". >2014-09-25 18:05:12,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_LOG_TO_CONSOLE" is "false". >2014-09-25 18:05:12,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_MANUAL" is "/usr/share/doc/rhevm-doc/manual". >2014-09-25 18:05:12,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MAX" is "256m". >2014-09-25 18:05:12,483 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PERM_MIN" is "256m". >2014-09-25 18:05:12,495 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI" is "/etc/pki/ovirt-engine". >2014-09-25 18:05:12,495 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_CA" is "/etc/pki/ovirt-engine/ca.pem". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_CERT" is "/etc/pki/ovirt-engine/certs/engine.cer". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE" is "/etc/pki/ovirt-engine/keys/engine.p12". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_ALIAS" is "1". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_ENGINE_STORE_PASSWORD" is "***". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE" is "/etc/pki/ovirt-engine/.truststore". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PKI_TRUST_STORE_PASSWORD" is "***". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROPERTIES" is " jsse.enableSNIExtension=false "java.security.krb5.conf=/etc/ovirt-engine/krb5.conf"". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_ENABLED" is "true". >2014-09-25 18:05:12,496 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTPS_PORT" is "443". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_PROXY_HTTP_PORT" is "80". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_BASE_URL" is "". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_DASHBOARD_URL" is "/flow.html?viewAsDashboardFrame=true". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_PROXY_URL" is "/ovirt/reports-interface". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_READ_TIMEOUT" is "". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_CHAIN" is "true". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_REPORTS_VERIFY_HOST" is "true". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_INTERVAL" is "1". >2014-09-25 18:05:12,497 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_STOP_TIME" is "10". >2014-09-25 18:05:12,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_TMP" is "/var/tmp/ovirt-engine". >2014-09-25 18:05:12,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_UP_MARK" is "/var/lib/ovirt-engine/engine.up". >2014-09-25 18:05:12,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_URI" is "/ovirt-engine". >2014-09-25 18:05:12,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USER" is "ovirt". >2014-09-25 18:05:12,498 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_USR" is "/usr/share/ovirt-engine". >2014-09-25 18:05:12,511 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VAR" is "/var/lib/ovirt-engine". >2014-09-25 18:05:12,512 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "ENGINE_VERBOSE_GC" is "false". >2014-09-25 18:05:12,512 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_HOME" is "/usr/share/jbossas". >2014-09-25 18:05:12,512 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "JBOSS_RUNTIME" is "/var/lib/ovirt-engine/jboss_runtime". >2014-09-25 18:05:12,512 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 38) Value of property "SENSITIVE_KEYS" is ",ENGINE_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD". >2014-09-25 18:05:22,179 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 44) Start initializing Backend >2014-09-25 18:05:22,564 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 44) Loaded file /etc/ovirt-engine/osinfo.conf.d/00-defaults.properties >2014-09-25 18:05:22,705 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 44) Running ovirt-engine 3.5.0-0.13.beta.el6ev >2014-09-25 18:05:22,705 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 44) Start initializing dictionaries >2014-09-25 18:05:22,708 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 44) Finished initializing dictionaries >2014-09-25 18:05:22,708 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Start initializing AuditLogCleanupManager >2014-09-25 18:05:22,714 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Setting audit cleanup manager to run at: 35 35 3 * * ? >2014-09-25 18:05:22,751 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Finished initializing AuditLogCleanupManager >2014-09-25 18:05:22,751 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Start initializing CommandEntityCleanupManager >2014-09-25 18:05:22,754 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Setting command entity cleanup manager to run at: 35 35 3 * * ? >2014-09-25 18:05:22,755 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 44) Finished initializing CommandEntityCleanupManager >2014-09-25 18:05:22,757 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 44) Start initializing TagsDirector >2014-09-25 18:05:22,787 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 44) Tag root added to tree >2014-09-25 18:05:22,801 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 44) Finished initializing TagsDirector >2014-09-25 18:05:22,802 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 44) Start initializing IsoDomainListSyncronizer >2014-09-25 18:05:22,809 INFO [org.ovirt.engine.core.bll.IsoDomainListSyncronizer] (ServerService Thread Pool -- 44) Finished initializing IsoDomainListSyncronizer >2014-09-25 18:05:22,852 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 44) Completed initializing handlers >2014-09-25 18:05:22,881 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 44) Start initializing ErrorTranslatorImpl >2014-09-25 18:05:22,895 WARN [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 44) Code MAC_ADDRESS_IS_IN_USE appears more than once in string table. >2014-09-25 18:05:22,896 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 44) Finished initializing ErrorTranslatorImpl >2014-09-25 18:05:22,897 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 44) Start initializing ErrorTranslatorImpl >2014-09-25 18:05:22,897 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 44) Finished initializing ErrorTranslatorImpl >2014-09-25 18:05:22,897 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 44) Mark incomplete jobs as UNKNOWN >2014-09-25 18:05:22,933 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 44) Start initializing JobRepositoryCleanupManager >2014-09-25 18:05:22,934 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 44) Finished initializing JobRepositoryCleanupManager >2014-09-25 18:05:22,934 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 44) Start initializing AutoRecoveryManager >2014-09-25 18:05:22,935 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 44) Finished initializing AutoRecoveryManager >2014-09-25 18:05:22,937 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 44) Start initializing ExecutionMessageDirector >2014-09-25 18:05:22,938 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 44) Finished initializing ExecutionMessageDirector >2014-09-25 18:05:22,985 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Loading extension 'builtin-authn-internal' >2014-09-25 18:05:22,986 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Extension 'builtin-authn-internal' loaded >2014-09-25 18:05:22,988 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Loading extension 'internal' >2014-09-25 18:05:22,991 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Extension 'internal' loaded >2014-09-25 18:05:22,992 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Initializing extension 'builtin-authn-internal' >2014-09-25 18:05:22,992 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Extension 'builtin-authn-internal' initialized >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Initializing extension 'internal' >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Extension 'internal' initialized >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Start of enabled extensions list >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Instance name: 'builtin-authn-internal', Extension name: 'Internal Authn (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) Instance name: 'internal', Extension name: 'Internal Authz (Built-in)', Version: 'N/A', Notes: '', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: 'N/A', Initialized: 'true' >2014-09-25 18:05:22,993 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 44) End of enabled extensions list >2014-09-25 18:05:23,105 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 40) Start initializing DbUserCacheManager >2014-09-25 18:05:23,112 INFO [org.ovirt.engine.core.bll.aaa.DbUserCacheManager] (ServerService Thread Pool -- 40) Finished initializing DbUserCacheManager >2014-09-25 18:05:23,139 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 40) Initialization of AsyncTaskManager completed successfully. >2014-09-25 18:05:23,141 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Start initializing ResourceManager >2014-09-25 18:05:23,236 INFO [org.ovirt.engine.core.bll.aaa.SyncUsers] (DefaultQuartzScheduler_Worker-1) Principal admin::internal synchronized >2014-09-25 18:05:23,259 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Entered VdsManager constructor >2014-09-25 18:05:23,312 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) Initialize vdsBroker (slot-7.rhev.lab.eng.brq.redhat.com,54,321) >2014-09-25 18:05:23,518 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) VDS 96620dcc-2509-48fe-841b-85247771c4fb was added to the Resource Manager >2014-09-25 18:05:23,550 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) Finished initializing ResourceManager >2014-09-25 18:05:23,558 INFO [org.ovirt.engine.core.bll.OvfDataUpdater] (ServerService Thread Pool -- 40) Initialization of OvfDataUpdater completed successfully. >2014-09-25 18:05:23,559 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Start scheduling to enable vds load balancer >2014-09-25 18:05:23,559 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Finished scheduling to enable vds load balancer >2014-09-25 18:05:23,560 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Start HA Reservation check >2014-09-25 18:05:23,560 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Finished HA Reservation check >2014-09-25 18:05:23,563 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Start initializing MacPoolManagerRanges >2014-09-25 18:05:23,573 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init VM custom properties utilities >2014-09-25 18:05:23,573 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) Init device custom properties utilities >2014-09-25 18:05:23,582 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initializing Scheduling manager >2014-09-25 18:05:23,596 INFO [org.ovirt.engine.core.bll.network.MacPoolManagerRanges] (org.ovirt.thread.pool-7-thread-1) Finished initializing. Available MACs in pool: 63 >2014-09-25 18:05:23,630 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) External scheduler disabled, discovery skipped >2014-09-25 18:05:23,630 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) Initialized Scheduling manager >2014-09-25 18:05:23,631 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) Initializing DWH Heart Beat >2014-09-25 18:05:23,635 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) DWH Heart Beat initialized >2014-09-25 18:05:26,668 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-25 18:05:27,337 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) START, GetHardwareInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vds=Host[blade,96620dcc-2509-48fe-841b-85247771c4fb]), log id: 50ceb909 >2014-09-25 18:05:27,346 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler_Worker-6) FINISH, GetHardwareInfoVDSCommand, log id: 50ceb909 >2014-09-25 18:05:27,668 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (DefaultQuartzScheduler_Worker-6) Initializing Host: blade >2014-09-25 18:05:27,852 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler_Worker-6) [195b8c18] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 96620dcc-2509-48fe-841b-85247771c4fb Type: VDS >2014-09-25 18:05:27,907 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-4) domain fc587b6a-bac3-4668-99de-7f6fd6acce7f:ISO_DOMAIN in problem. vds: blade >2014-09-25 18:05:27,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-4) Host 96620dcc-2509-48fe-841b-85247771c4fb has reported new storage access problem to the following domains fc587b6a-bac3-4668-99de-7f6fd6acce7f marking it for storage connections and pool metadata refresh (report id: 3d34823f-e451-4b7e-9c82-0d534bc1183c) >2014-09-25 18:05:33,585 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-25 18:05:33,598 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-25 18:05:33,603 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain fc587b6a-bac3-4668-99de-7f6fd6acce7f status from Active to Unknown, reason : null >2014-09-25 18:05:33,605 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : null >2014-09-25 18:05:33,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: VDS_HIGH_NETWORK_USE not exist in string table >2014-09-25 18:05:33,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_REMOVE_VM not exist in string table >2014-09-25 18:05:33,625 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_RUN_UNLOCK_ENTITY_SCRIPT not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: VDS_NETWORK_MTU_DIFFER_FROM_LOGICAL_NETWORK not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: STORAGE_ACTIVATE_ASYNC not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_ADDED_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_ADD_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_REMOVED_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,626 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_REMOVE_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_UPDATED_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_UPDATE_DISK_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_ADDED_CPU_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_ADD_CPU_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_REMOVED_CPU_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_REMOVE_CPU_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_UPDATED_CPU_PROFILE not exist in string table >2014-09-25 18:05:33,627 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] AuditLogType: USER_FAILED_TO_UPDATE_CPU_PROFILE not exist in string table >2014-09-25 18:05:34,009 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Correlation ID: 2a25f5c9, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting status to Non Responsive. >2014-09-25 18:05:34,114 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] hostFromVds::selectedVds - blade, spmStatus SPM, storage pool Default >2014-09-25 18:05:34,121 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Initialize Irs proxy from vds: slot-7.rhev.lab.eng.brq.redhat.com >2014-09-25 18:05:34,130 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host blade (Address: slot-7.rhev.lab.eng.brq.redhat.com). >2014-09-25 18:05:34,138 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) Connecting to slot-7.rhev.lab.eng.brq.redhat.com/10.34.63.137 >2014-09-25 18:05:34,171 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false), log id: 2d454403 >2014-09-25 18:05:34,320 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-0000000001ad >2014-09-25 18:05:34,327 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] START, HSMGetAllTasksInfoVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb), log id: 576fbf8f >2014-09-25 18:05:34,338 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 576fbf8f >2014-09-25 18:05:34,338 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 2d454403 >2014-09-25 18:05:34,339 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-7-thread-7) [2a25f5c9] Discovered no tasks on Storage Pool Default >2014-09-25 18:05:53,568 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-64) Running storage connections refresh for hosts [96620dcc-2509-48fe-841b-85247771c4fb] >2014-09-25 18:05:53,589 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-7-thread-16) [140e1b6c] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-25 18:05:53,629 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-16) [140e1b6c] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageType = NFS, connectionList = [{ id: 6d77ae90-0a92-4f76-9980-cb7eceb7df64, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35export, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 58d5672c >2014-09-25 18:06:53,698 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-16) [140e1b6c] FINISH, ConnectStorageServerVDSCommand, return: {6d77ae90-0a92-4f76-9980-cb7eceb7df64=0, d7af8c51-78e6-44db-88d2-f765dc508251=0, 83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 58d5672c >2014-09-25 18:06:53,700 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-7-thread-16) [140e1b6c] Host blade storage connection was succeeded >2014-09-25 18:06:53,701 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-64) Submitting to the event queue pool refresh for hosts [96620dcc-2509-48fe-841b-85247771c4fb] >2014-09-25 18:06:53,703 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-46) Running storage pool metadata refresh for hosts {1} >2014-09-25 18:06:53,707 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-47) START, ConnectStoragePoolVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, vdsId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad, masterVersion = 1), log id: 858fd39 >2014-09-25 18:06:54,933 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-14) Failed in SpmStatusVDS method >2014-09-25 18:06:54,962 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-14) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:06:54,970 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (DefaultQuartzScheduler_Worker-14) [36a2470] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-0000000001ad Type: StoragePool >2014-09-25 18:06:54,973 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-14) [36a2470] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain eb633636-3019-47a9-9f5a-fc81f8dfd7dc status from Active to Unknown, reason : null >2014-09-25 18:06:54,974 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-14) [36a2470] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain fc587b6a-bac3-4668-99de-7f6fd6acce7f status from Active to Unknown, reason : null >2014-09-25 18:06:54,974 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (DefaultQuartzScheduler_Worker-14) [36a2470] Storage Pool 00000002-0002-0002-0002-0000000001ad - Updating Storage Domain 3573f707-d0c4-4dc9-b09f-dca9a3506a4f status from Active to Unknown, reason : null >2014-09-25 18:06:54,995 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-14) [36a2470] Correlation ID: 36a2470, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host blade, Error: General Exception). >2014-09-25 18:06:55,038 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-14) [36a2470] Failed in SpmStatusVDS method >2014-09-25 18:06:55,040 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-14) [36a2470] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:06:55,040 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-14) [36a2470] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:05,084 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:07:05,086 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:05,086 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:15,131 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Failed in SpmStatusVDS method >2014-09-25 18:07:15,133 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:15,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:25,166 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in SpmStatusVDS method >2014-09-25 18:07:25,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:25,168 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:35,197 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Failed in SpmStatusVDS method >2014-09-25 18:07:35,199 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:35,200 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:45,228 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Failed in SpmStatusVDS method >2014-09-25 18:07:45,231 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:45,231 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:07:53,974 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-7-thread-47) FINISH, ConnectStoragePoolVDSCommand, log id: 858fd39 >2014-09-25 18:07:55,259 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) Failed in SpmStatusVDS method >2014-09-25 18:07:55,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-40) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:07:55,261 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-40) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:05,293 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-61) Failed in SpmStatusVDS method >2014-09-25 18:08:05,296 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-61) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:05,297 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-61) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:15,348 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Failed in SpmStatusVDS method >2014-09-25 18:08:15,350 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:15,350 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:25,376 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Failed in SpmStatusVDS method >2014-09-25 18:08:25,382 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:25,382 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:35,436 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Failed in SpmStatusVDS method >2014-09-25 18:08:35,438 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:35,438 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:45,473 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Failed in SpmStatusVDS method >2014-09-25 18:08:45,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:45,481 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:08:55,515 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) Failed in SpmStatusVDS method >2014-09-25 18:08:55,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:08:55,517 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-87) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:05,551 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Failed in SpmStatusVDS method >2014-09-25 18:09:05,553 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:05,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:15,586 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:09:15,588 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:15,588 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:25,619 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) Failed in SpmStatusVDS method >2014-09-25 18:09:25,621 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:25,621 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:35,654 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Failed in SpmStatusVDS method >2014-09-25 18:09:35,656 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:35,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:45,693 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Failed in SpmStatusVDS method >2014-09-25 18:09:45,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:45,695 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:09:55,730 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-35) Failed in SpmStatusVDS method >2014-09-25 18:09:55,732 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-35) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:09:55,732 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-35) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:05,757 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-57) Failed in SpmStatusVDS method >2014-09-25 18:10:05,759 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-57) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:05,759 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-57) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:15,796 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) Failed in SpmStatusVDS method >2014-09-25 18:10:15,798 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:15,798 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-81) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:25,831 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-13) Failed in SpmStatusVDS method >2014-09-25 18:10:25,833 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-13) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:25,833 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-13) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:27,912 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) starting processDomainRecovery for domain fc587b6a-bac3-4668-99de-7f6fd6acce7f:ISO_DOMAIN >2014-09-25 18:10:27,989 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) Domain fc587b6a-bac3-4668-99de-7f6fd6acce7f:ISO_DOMAIN was reported by all hosts in status UP as problematic. Moving the domain to NonOperational. >2014-09-25 18:10:28,057 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Lock Acquired to object EngineLock [exclusiveLocks= key: fc587b6a-bac3-4668-99de-7f6fd6acce7f value: STORAGE >, sharedLocks= ] >2014-09-25 18:10:28,079 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Running command: DeactivateStorageDomainCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:10:28,124 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Lock freed to object EngineLock [exclusiveLocks= key: fc587b6a-bac3-4668-99de-7f6fd6acce7f value: STORAGE >, sharedLocks= ] >2014-09-25 18:10:28,159 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Correlation ID: 2b596b16, Job ID: 6c02e10b-5b6f-45a8-a773-d790da8e0708, Call Stack: null, Custom Event ID: -1, Message: Storage Domain ISO_DOMAIN (Data Center Default) was deactivated by system because it's not visible by any of the hosts. >2014-09-25 18:10:28,179 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Removing vds [] from the domain in maintenance cache >2014-09-25 18:10:28,180 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-23) [2b596b16] Removing host(s) [] from hosts unseen domain report cache >2014-09-25 18:10:35,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Failed in SpmStatusVDS method >2014-09-25 18:10:35,867 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-39) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:35,867 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-39) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:41,191 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-30) Host 96620dcc-2509-48fe-841b-85247771c4fb no longer storage access problem to any relevant domain clearing its report (report id: 3d34823f-e451-4b7e-9c82-0d534bc1183c) >2014-09-25 18:10:43,986 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-56) No hosts has reported storage access problem to domains, clearing the handled hosts reports map >2014-09-25 18:10:45,908 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Failed in SpmStatusVDS method >2014-09-25 18:10:45,910 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:45,910 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:10:55,947 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Failed in SpmStatusVDS method >2014-09-25 18:10:55,949 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:10:55,949 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:05,986 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Failed in SpmStatusVDS method >2014-09-25 18:11:05,987 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:05,987 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:16,017 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:11:16,019 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:16,019 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:26,059 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Failed in SpmStatusVDS method >2014-09-25 18:11:26,061 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:26,065 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:36,096 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Failed in SpmStatusVDS method >2014-09-25 18:11:36,097 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:36,098 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:46,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Failed in SpmStatusVDS method >2014-09-25 18:11:46,133 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:46,136 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:11:56,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:11:56,170 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:11:56,170 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:06,204 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-56) Failed in SpmStatusVDS method >2014-09-25 18:12:06,205 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-56) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:06,206 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-56) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:16,239 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Failed in SpmStatusVDS method >2014-09-25 18:12:16,240 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:16,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:26,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Failed in SpmStatusVDS method >2014-09-25 18:12:26,277 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:26,277 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:36,311 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Failed in SpmStatusVDS method >2014-09-25 18:12:36,312 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:36,312 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:46,343 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Failed in SpmStatusVDS method >2014-09-25 18:12:46,345 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:46,345 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:12:56,375 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Failed in SpmStatusVDS method >2014-09-25 18:12:56,376 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:12:56,376 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:06,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-3) Failed in SpmStatusVDS method >2014-09-25 18:13:06,409 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-3) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:06,409 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-3) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:16,441 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Failed in SpmStatusVDS method >2014-09-25 18:13:16,443 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:16,443 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:26,484 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Failed in SpmStatusVDS method >2014-09-25 18:13:26,485 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:26,485 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:36,517 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in SpmStatusVDS method >2014-09-25 18:13:36,518 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:36,518 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:46,548 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Failed in SpmStatusVDS method >2014-09-25 18:13:46,552 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:46,552 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:13:56,589 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Failed in SpmStatusVDS method >2014-09-25 18:13:56,591 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:13:56,591 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:06,626 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Failed in SpmStatusVDS method >2014-09-25 18:14:06,628 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:06,628 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:16,661 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Failed in SpmStatusVDS method >2014-09-25 18:14:16,662 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:16,663 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:26,695 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Failed in SpmStatusVDS method >2014-09-25 18:14:26,697 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:26,697 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:36,728 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-27) Failed in SpmStatusVDS method >2014-09-25 18:14:36,730 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-27) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:36,730 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:46,759 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Failed in SpmStatusVDS method >2014-09-25 18:14:46,761 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:46,761 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:14:56,791 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Failed in SpmStatusVDS method >2014-09-25 18:14:56,792 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:14:56,792 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:00,013 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-82) Autorecovering 1 storage domains >2014-09-25 18:15:00,014 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-82) Autorecovering storage domains id: fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-25 18:15:00,020 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-82) [2b204d5b] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: Storage >2014-09-25 18:15:00,020 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-82) [2b204d5b] ConnectDomainToStorage. Before Connect all hosts to pool. Time:9/25/14 6:15 PM >2014-09-25 18:15:00,091 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-45) [79a5b98f] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:15:00,095 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-45) [79a5b98f] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3e621210 >2014-09-25 18:15:06,829 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Failed in SpmStatusVDS method >2014-09-25 18:15:06,830 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:06,830 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:16,880 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-23) Failed in SpmStatusVDS method >2014-09-25 18:15:16,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-23) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:16,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-23) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:27,014 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Failed in SpmStatusVDS method >2014-09-25 18:15:27,016 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:27,016 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:37,045 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in SpmStatusVDS method >2014-09-25 18:15:37,047 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:37,047 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:47,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-1) Failed in SpmStatusVDS method >2014-09-25 18:15:47,079 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-1) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:47,080 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-1) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:15:57,111 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Failed in SpmStatusVDS method >2014-09-25 18:15:57,113 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:15:57,113 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:00,108 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-45) [79a5b98f] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 3e621210 >2014-09-25 18:16:00,112 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-82) [2b204d5b] ConnectDomainToStorage. After Connect all hosts to pool. Time:9/25/14 6:16 PM >2014-09-25 18:16:07,146 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Failed in SpmStatusVDS method >2014-09-25 18:16:07,148 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:07,148 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:17,182 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) Failed in SpmStatusVDS method >2014-09-25 18:16:17,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-72) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:17,184 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-72) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:27,215 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-5) Failed in SpmStatusVDS method >2014-09-25 18:16:27,217 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-5) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:27,217 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-5) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:37,243 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Failed in SpmStatusVDS method >2014-09-25 18:16:37,244 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:37,245 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:47,275 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) Failed in SpmStatusVDS method >2014-09-25 18:16:47,276 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:47,276 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:16:57,317 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Failed in SpmStatusVDS method >2014-09-25 18:16:57,319 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:16:57,319 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:07,350 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-2) Failed in SpmStatusVDS method >2014-09-25 18:17:07,352 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-2) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:07,352 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-2) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:17,382 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Failed in SpmStatusVDS method >2014-09-25 18:17:17,383 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:17,383 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:27,414 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) Failed in SpmStatusVDS method >2014-09-25 18:17:27,415 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-50) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:27,416 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-50) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:37,446 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Failed in SpmStatusVDS method >2014-09-25 18:17:37,448 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-73) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:37,448 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-73) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:47,484 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Failed in SpmStatusVDS method >2014-09-25 18:17:47,486 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:47,486 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:17:57,526 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-25) Failed in SpmStatusVDS method >2014-09-25 18:17:57,527 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-25) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:17:57,527 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-25) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:07,557 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) Failed in SpmStatusVDS method >2014-09-25 18:18:07,559 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:07,559 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:17,599 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Failed in SpmStatusVDS method >2014-09-25 18:18:17,600 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:17,600 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:27,631 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Failed in SpmStatusVDS method >2014-09-25 18:18:27,632 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-99) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:27,633 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-99) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:37,663 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Failed in SpmStatusVDS method >2014-09-25 18:18:37,664 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:37,665 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:47,703 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Failed in SpmStatusVDS method >2014-09-25 18:18:47,704 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:47,704 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:18:57,735 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Failed in SpmStatusVDS method >2014-09-25 18:18:57,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:18:57,737 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:07,768 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Failed in SpmStatusVDS method >2014-09-25 18:19:07,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:07,770 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:17,800 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Failed in SpmStatusVDS method >2014-09-25 18:19:17,801 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:17,801 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:27,832 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Failed in SpmStatusVDS method >2014-09-25 18:19:27,834 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:27,834 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:37,866 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) Failed in SpmStatusVDS method >2014-09-25 18:19:37,867 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:37,867 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-67) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:47,898 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Failed in SpmStatusVDS method >2014-09-25 18:19:47,899 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:47,899 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:19:57,929 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Failed in SpmStatusVDS method >2014-09-25 18:19:57,931 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:19:57,931 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:00,010 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-25) Autorecovering 1 storage domains >2014-09-25 18:20:00,010 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-25) Autorecovering storage domains id: fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-25 18:20:00,012 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-25) [3295f89f] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: Storage >2014-09-25 18:20:00,014 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-25) [3295f89f] ConnectDomainToStorage. Before Connect all hosts to pool. Time:9/25/14 6:20 PM >2014-09-25 18:20:00,028 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-38) [54e93692] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:20:00,033 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [54e93692] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 40ae8610 >2014-09-25 18:20:07,973 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-41) Failed in SpmStatusVDS method >2014-09-25 18:20:07,974 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-41) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:07,974 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-41) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:18,008 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) Failed in SpmStatusVDS method >2014-09-25 18:20:18,009 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:18,009 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:28,051 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Failed in SpmStatusVDS method >2014-09-25 18:20:28,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:28,053 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:38,085 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-19) Failed in SpmStatusVDS method >2014-09-25 18:20:38,087 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-19) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:38,087 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:48,117 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-43) Failed in SpmStatusVDS method >2014-09-25 18:20:48,119 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-43) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:48,119 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-43) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:20:58,148 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-69) Failed in SpmStatusVDS method >2014-09-25 18:20:58,149 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-69) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:20:58,149 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-69) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:00,045 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-38) [54e93692] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 40ae8610 >2014-09-25 18:21:00,049 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-25) [3295f89f] ConnectDomainToStorage. After Connect all hosts to pool. Time:9/25/14 6:21 PM >2014-09-25 18:21:08,179 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Failed in SpmStatusVDS method >2014-09-25 18:21:08,180 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:08,180 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:18,213 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Failed in SpmStatusVDS method >2014-09-25 18:21:18,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:18,214 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:28,261 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Failed in SpmStatusVDS method >2014-09-25 18:21:28,262 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:28,262 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:38,296 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) Failed in SpmStatusVDS method >2014-09-25 18:21:38,298 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-67) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:38,298 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-67) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:48,330 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) Failed in SpmStatusVDS method >2014-09-25 18:21:48,332 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:48,332 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:21:58,370 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Failed in SpmStatusVDS method >2014-09-25 18:21:58,371 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-15) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:21:58,371 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-15) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:08,403 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-38) Failed in SpmStatusVDS method >2014-09-25 18:22:08,404 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-38) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:08,404 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-38) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:18,433 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) Failed in SpmStatusVDS method >2014-09-25 18:22:18,435 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:18,435 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-63) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:28,463 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) Failed in SpmStatusVDS method >2014-09-25 18:22:28,464 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-89) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:28,464 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-89) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:38,669 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-16) Failed in SpmStatusVDS method >2014-09-25 18:22:38,671 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-16) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:38,671 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-16) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:48,708 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-38) Failed in SpmStatusVDS method >2014-09-25 18:22:48,709 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-38) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:48,710 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-38) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:22:58,739 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) Failed in SpmStatusVDS method >2014-09-25 18:22:58,740 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-65) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:22:58,741 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-65) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:08,772 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-88) Failed in SpmStatusVDS method >2014-09-25 18:23:08,773 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-88) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:08,773 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-88) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:18,809 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Failed in SpmStatusVDS method >2014-09-25 18:23:18,811 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-10) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:18,811 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-10) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:28,843 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-36) Failed in SpmStatusVDS method >2014-09-25 18:23:28,845 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-36) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:28,845 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-36) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:38,879 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) Failed in SpmStatusVDS method >2014-09-25 18:23:38,881 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:38,881 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-63) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:48,918 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-90) Failed in SpmStatusVDS method >2014-09-25 18:23:48,919 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-90) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:48,919 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-90) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:23:58,947 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) Failed in SpmStatusVDS method >2014-09-25 18:23:58,948 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:23:58,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-12) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:08,977 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) Failed in SpmStatusVDS method >2014-09-25 18:24:08,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-37) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:08,978 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-37) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:19,009 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-57) Failed in SpmStatusVDS method >2014-09-25 18:24:19,010 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-57) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:19,010 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-57) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:29,041 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Failed in SpmStatusVDS method >2014-09-25 18:24:29,042 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:29,042 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:39,077 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Failed in SpmStatusVDS method >2014-09-25 18:24:39,079 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:39,079 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:49,114 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:24:49,115 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:49,115 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:24:59,150 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Failed in SpmStatusVDS method >2014-09-25 18:24:59,151 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:24:59,151 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:00,018 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-63) Autorecovering 1 storage domains >2014-09-25 18:25:00,018 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-63) Autorecovering storage domains id: fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-25 18:25:00,023 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-63) [612d04f3] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: Storage >2014-09-25 18:25:00,024 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-63) [612d04f3] ConnectDomainToStorage. Before Connect all hosts to pool. Time:9/25/14 6:25 PM >2014-09-25 18:25:00,036 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-45) [32da4c98] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:25:00,042 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-45) [32da4c98] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 3cb465a >2014-09-25 18:25:09,182 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Failed in SpmStatusVDS method >2014-09-25 18:25:09,184 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:09,184 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:19,216 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-5) Failed in SpmStatusVDS method >2014-09-25 18:25:19,218 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-5) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:19,218 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-5) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:29,250 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) Failed in SpmStatusVDS method >2014-09-25 18:25:29,251 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-22) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:29,251 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-22) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:39,282 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) Failed in SpmStatusVDS method >2014-09-25 18:25:39,284 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-59) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:39,284 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-59) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:49,313 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Failed in SpmStatusVDS method >2014-09-25 18:25:49,314 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-86) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:49,314 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-86) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:25:59,366 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) Failed in SpmStatusVDS method >2014-09-25 18:25:59,367 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-12) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:25:59,367 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-12) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:00,056 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-45) [32da4c98] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 3cb465a >2014-09-25 18:26:00,059 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-63) [612d04f3] ConnectDomainToStorage. After Connect all hosts to pool. Time:9/25/14 6:26 PM >2014-09-25 18:26:09,399 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Failed in SpmStatusVDS method >2014-09-25 18:26:09,400 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:09,400 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:19,432 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Failed in SpmStatusVDS method >2014-09-25 18:26:19,434 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-60) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:19,434 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-60) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:29,470 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) Failed in SpmStatusVDS method >2014-09-25 18:26:29,472 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-87) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:29,472 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-87) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:39,507 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-13) Failed in SpmStatusVDS method >2014-09-25 18:26:39,508 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-13) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:39,508 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-13) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:49,553 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Failed in SpmStatusVDS method >2014-09-25 18:26:49,554 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:49,554 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:26:59,585 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) Failed in SpmStatusVDS method >2014-09-25 18:26:59,587 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-62) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:26:59,588 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-62) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:09,617 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-84) Failed in SpmStatusVDS method >2014-09-25 18:27:09,618 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-84) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:09,618 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-84) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:19,651 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) Failed in SpmStatusVDS method >2014-09-25 18:27:19,653 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-11) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:19,653 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-11) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:29,684 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Failed in SpmStatusVDS method >2014-09-25 18:27:29,686 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-64) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:29,686 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-64) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:39,715 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) Failed in SpmStatusVDS method >2014-09-25 18:27:39,717 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-58) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:39,717 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-58) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:49,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Failed in SpmStatusVDS method >2014-09-25 18:27:49,748 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:49,748 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:27:59,779 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Failed in SpmStatusVDS method >2014-09-25 18:27:59,781 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-9) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:27:59,781 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-9) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:09,815 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Failed in SpmStatusVDS method >2014-09-25 18:28:09,816 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-34) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:09,816 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-34) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:19,848 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Failed in SpmStatusVDS method >2014-09-25 18:28:19,849 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:19,849 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:29,883 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Failed in SpmStatusVDS method >2014-09-25 18:28:29,884 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-85) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:29,884 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-85) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:39,916 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Failed in SpmStatusVDS method >2014-09-25 18:28:39,917 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:39,917 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:49,947 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Failed in SpmStatusVDS method >2014-09-25 18:28:49,948 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:49,948 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:28:59,978 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Failed in SpmStatusVDS method >2014-09-25 18:28:59,983 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-55) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:28:59,983 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:10,018 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) Failed in SpmStatusVDS method >2014-09-25 18:29:10,019 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-81) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:29:10,020 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-81) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:20,049 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-3) Failed in SpmStatusVDS method >2014-09-25 18:29:20,052 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-3) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:29:20,052 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-3) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:30,081 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Failed in SpmStatusVDS method >2014-09-25 18:29:30,082 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:29:30,082 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:30,286 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-1) Running command: LoginUserCommand internal: false. >2014-09-25 18:29:30,316 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-1) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-25 18:29:37,551 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-25 18:29:37,551 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-3) Invalid locale found in configuration: >2014-09-25 18:29:40,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) Failed in SpmStatusVDS method >2014-09-25 18:29:40,111 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:29:40,111 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:44,895 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin descriptor [/usr/share/ovirt-engine/ui-plugins/redhat_support_plugin_rhev.json] >2014-09-25 18:29:44,896 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (ajp-/127.0.0.1:8702-5) Reading UI plugin configuration [/etc/ovirt-engine/ui-plugins/redhat_support_plugin_rhev-config.json] >2014-09-25 18:29:44,910 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-25 18:29:44,910 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp-/127.0.0.1:8702-5) Invalid locale found in configuration: >2014-09-25 18:29:50,141 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Failed in SpmStatusVDS method >2014-09-25 18:29:50,142 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:29:50,142 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:29:54,568 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp-/127.0.0.1:8702-8) Running command: LoginAdminUserCommand internal: false. >2014-09-25 18:29:54,580 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp-/127.0.0.1:8702-8) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. >2014-09-25 18:29:56,635 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp-/127.0.0.1:8702-3) Running command: LoginUserCommand internal: false. >2014-09-25 18:30:00,007 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-3) Autorecovering 1 storage domains >2014-09-25 18:30:00,007 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-3) Autorecovering storage domains id: fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-25 18:30:00,008 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-3) [12d3bd0c] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: Storage >2014-09-25 18:30:00,008 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-3) [12d3bd0c] ConnectDomainToStorage. Before Connect all hosts to pool. Time:9/25/14 6:30 PM >2014-09-25 18:30:00,017 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-20) [641743af] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:30:00,020 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-20) [641743af] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1b04b43a >2014-09-25 18:30:00,168 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Failed in SpmStatusVDS method >2014-09-25 18:30:00,169 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-4) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:00,169 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-4) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:30:10,202 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Failed in SpmStatusVDS method >2014-09-25 18:30:10,203 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-28) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:10,203 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-28) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:30:20,234 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-52) Failed in SpmStatusVDS method >2014-09-25 18:30:20,236 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-52) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:20,236 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-52) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:30:30,266 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Failed in SpmStatusVDS method >2014-09-25 18:30:30,267 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-80) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:30,267 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-80) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:30:40,298 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Failed in SpmStatusVDS method >2014-09-25 18:30:40,299 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-8) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:40,299 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-8) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:30:50,328 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-32) Failed in SpmStatusVDS method >2014-09-25 18:30:50,330 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-32) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:30:50,330 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-32) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:00,031 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-20) [641743af] FINISH, ConnectStorageServerVDSCommand, return: {83fd005b-3471-44cb-99d9-bc14b3dc1ae8=0}, log id: 1b04b43a >2014-09-25 18:31:00,035 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-3) [12d3bd0c] ConnectDomainToStorage. After Connect all hosts to pool. Time:9/25/14 6:31 PM >2014-09-25 18:31:00,357 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Failed in SpmStatusVDS method >2014-09-25 18:31:00,358 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-54) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:00,359 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-54) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:10,387 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Failed in SpmStatusVDS method >2014-09-25 18:31:10,388 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-76) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:10,388 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-76) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:20,420 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-100) Failed in SpmStatusVDS method >2014-09-25 18:31:20,422 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-100) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:20,422 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-100) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:30,454 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Failed in SpmStatusVDS method >2014-09-25 18:31:30,456 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-33) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:30,456 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-33) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:31,857 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-8) [2ed36fcb] Lock Acquired to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:31:31,899 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:31:31,908 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Lock freed to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:31:31,909 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/25/14 6:31 PM >2014-09-25 18:31:31,915 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-17) [2c8fefd0] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:31:31,921 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-17) [2c8fefd0] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1a06618 >2014-09-25 18:31:31,940 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-17) [2c8fefd0] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 1a06618 >2014-09-25 18:31:31,944 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc), log id: 272a2d21 >2014-09-25 18:31:32,130 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Failed in SpmStatusVDS method >2014-09-25 18:31:32,132 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:32,132 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:32,133 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] FINISH, ActivateStorageDomainVDSCommand, log id: 272a2d21 >2014-09-25 18:31:32,133 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: Cannot allocate IRS server (Failed with error IRS_REPOSITORY_NOT_FOUND and code 5009) >2014-09-25 18:31:32,166 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Command [id=908292e1-d933-4d42-9f7a-809965160a5e]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, status=Unknown]. >2014-09-25 18:31:32,176 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-15) [2ed36fcb] Correlation ID: 2ed36fcb, Job ID: 4de90ceb-28ba-4c21-8bdd-b2e152df16a5, Call Stack: null, Custom Event ID: -1, Message: Failed to activate Storage Domain master (Data Center Default) by admin >2014-09-25 18:31:40,488 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) Failed in SpmStatusVDS method >2014-09-25 18:31:40,489 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-53) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:40,489 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-53) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:31:50,519 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Failed in SpmStatusVDS method >2014-09-25 18:31:50,520 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-79) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:31:50,520 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-79) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:00,552 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-7) Failed in SpmStatusVDS method >2014-09-25 18:32:00,553 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-7) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:00,553 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-7) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:10,583 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) Failed in SpmStatusVDS method >2014-09-25 18:32:10,584 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-29) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:10,584 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-29) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:20,615 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) Failed in SpmStatusVDS method >2014-09-25 18:32:20,616 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-48) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:20,616 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-48) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:30,647 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) Failed in SpmStatusVDS method >2014-09-25 18:32:30,648 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-75) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:30,649 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-75) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:36,571 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-5) [3d98c9c0] Lock Acquired to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:32:36,599 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:32:36,604 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Lock freed to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:32:36,605 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/25/14 6:32 PM >2014-09-25 18:32:36,610 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-46) [75157632] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:32:36,616 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-46) [75157632] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7256a441 >2014-09-25 18:32:36,636 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-46) [75157632] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 7256a441 >2014-09-25 18:32:36,638 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc), log id: 1a237b1c >2014-09-25 18:32:36,818 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Failed in SpmStatusVDS method >2014-09-25 18:32:36,820 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:36,820 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:36,820 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] FINISH, ActivateStorageDomainVDSCommand, log id: 1a237b1c >2014-09-25 18:32:36,821 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: Cannot allocate IRS server (Failed with error IRS_REPOSITORY_NOT_FOUND and code 5009) >2014-09-25 18:32:36,824 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Command [id=6fb8b6c6-3fea-460f-8aef-1318763b1607]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, status=Unknown]. >2014-09-25 18:32:36,831 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-49) [3d98c9c0] Correlation ID: 3d98c9c0, Job ID: 499bac58-3322-4ba3-b21b-bd8d7046fa90, Call Stack: null, Custom Event ID: -1, Message: Failed to activate Storage Domain master (Data Center Default) by admin >2014-09-25 18:32:40,678 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-1) Failed in SpmStatusVDS method >2014-09-25 18:32:40,680 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-1) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:40,680 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-1) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:32:50,712 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Failed in SpmStatusVDS method >2014-09-25 18:32:50,714 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:32:50,714 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:00,747 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Failed in SpmStatusVDS method >2014-09-25 18:33:00,749 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:00,749 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:10,795 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-74) Failed in SpmStatusVDS method >2014-09-25 18:33:10,797 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-74) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:10,797 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-74) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:20,840 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Failed in SpmStatusVDS method >2014-09-25 18:33:20,842 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:20,842 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:30,872 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-27) Failed in SpmStatusVDS method >2014-09-25 18:33:30,874 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-27) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:30,874 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-27) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:40,927 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Failed in SpmStatusVDS method >2014-09-25 18:33:40,928 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-51) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:40,928 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-51) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:33:50,957 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) Failed in SpmStatusVDS method >2014-09-25 18:33:50,958 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:33:50,958 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-77) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:00,992 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Failed in SpmStatusVDS method >2014-09-25 18:34:00,993 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-98) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:00,993 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-98) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:05,296 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp-/127.0.0.1:8702-9) [630b8040] Lock Acquired to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:34:05,365 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: eb633636-3019-47a9-9f5a-fc81f8dfd7dc Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:34:05,376 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Lock freed to object EngineLock [exclusiveLocks= key: eb633636-3019-47a9-9f5a-fc81f8dfd7dc value: STORAGE >, sharedLocks= ] >2014-09-25 18:34:05,377 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] ActivateStorage Domain. Before Connect all hosts to pool. Time:9/25/14 6:34 PM >2014-09-25 18:34:05,381 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-42) [1fb4d0a1] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:34:05,384 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-42) [1fb4d0a1] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: d7af8c51-78e6-44db-88d2-f765dc508251, connection: 10.34.63.202:/mnt/export/nfs/lv3/pbenas/35, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1eaa6844 >2014-09-25 18:34:05,435 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-42) [1fb4d0a1] FINISH, ConnectStorageServerVDSCommand, return: {d7af8c51-78e6-44db-88d2-f765dc508251=0}, log id: 1eaa6844 >2014-09-25 18:34:05,437 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] START, ActivateStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-0000000001ad, ignoreFailoverLimit = false, storageDomainId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc), log id: 68b0ee22 >2014-09-25 18:34:05,654 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Failed in SpmStatusVDS method >2014-09-25 18:34:05,656 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:05,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-7-thread-40) [630b8040] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:05,656 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] FINISH, ActivateStorageDomainVDSCommand, log id: 68b0ee22 >2014-09-25 18:34:05,656 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: Cannot allocate IRS server (Failed with error IRS_REPOSITORY_NOT_FOUND and code 5009) >2014-09-25 18:34:05,657 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-7-thread-40) [630b8040] Command [id=eda3c91b-94df-4b32-b03f-d98d113a13ed]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = 00000002-0002-0002-0002-0000000001ad, storageId = eb633636-3019-47a9-9f5a-fc81f8dfd7dc, status=Unknown]. >2014-09-25 18:34:05,665 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-7-thread-40) [630b8040] Correlation ID: 630b8040, Job ID: ea5935c4-54ec-4008-a54c-1442cb55ffea, Call Stack: null, Custom Event ID: -1, Message: Failed to activate Storage Domain master (Data Center Default) by admin >2014-09-25 18:34:11,043 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-19) Failed in SpmStatusVDS method >2014-09-25 18:34:11,044 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-19) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:11,045 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-19) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:21,076 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-42) Failed in SpmStatusVDS method >2014-09-25 18:34:21,078 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-42) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:21,078 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-42) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:31,109 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) Failed in SpmStatusVDS method >2014-09-25 18:34:31,111 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-77) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:31,111 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-77) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:41,171 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Failed in SpmStatusVDS method >2014-09-25 18:34:41,173 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-95) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:41,173 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-95) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:34:51,207 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Failed in SpmStatusVDS method >2014-09-25 18:34:51,209 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-17) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:34:51,209 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-17) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:35:00,008 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-40) Autorecovering 1 storage domains >2014-09-25 18:35:00,009 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-40) Autorecovering storage domains id: fc587b6a-bac3-4668-99de-7f6fd6acce7f >2014-09-25 18:35:00,011 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-40) [2ac97c33] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: fc587b6a-bac3-4668-99de-7f6fd6acce7f Type: Storage >2014-09-25 18:35:00,011 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-40) [2ac97c33] ConnectDomainToStorage. Before Connect all hosts to pool. Time:9/25/14 6:35 PM >2014-09-25 18:35:00,036 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-7-thread-19) [71007955] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN >2014-09-25 18:35:00,040 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-7-thread-19) [71007955] START, ConnectStorageServerVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 83fd005b-3471-44cb-99d9-bc14b3dc1ae8, connection: pb-rh35.rhev.lab.eng.brq.redhat.com:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 427284cf >2014-09-25 18:35:01,239 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Failed in SpmStatusVDS method >2014-09-25 18:35:01,240 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-45) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:35:01,240 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-45) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:35:11,288 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Failed in SpmStatusVDS method >2014-09-25 18:35:11,289 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-71) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:35:11,290 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-71) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:35:21,320 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Failed in SpmStatusVDS method >2014-09-25 18:35:21,321 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-93) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:35:21,322 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-93) hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:35:31,353 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Failed in SpmStatusVDS method >2014-09-25 18:35:31,354 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:35:31,354 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-20) [2a25f5c9] hostFromVds::selectedVds - blade, spmStatus returned null! >2014-09-25 18:35:41,448 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Failed in SpmStatusVDS method >2014-09-25 18:35:41,449 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (DefaultQuartzScheduler_Worker-46) Command SpmStatusVDSCommand(HostName = blade, HostId = 96620dcc-2509-48fe-841b-85247771c4fb, storagePoolId = 00000002-0002-0002-0002-0000000001ad) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = , code = 100 >2014-09-25 18:35:41,451 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-46) hostFromVds::selectedVds - blade, spmStatus returned null!
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1146648
:
941196
| 941197 |
941204